Skip to content

Instantly share code, notes, and snippets.

@VictorTaelin
Last active February 18, 2026 08:53
Show Gist options
  • Select an option

  • Save VictorTaelin/45440a737e47b872d7505c6cda27b6aa to your computer and use it in GitHub Desktop.

Select an option

Save VictorTaelin/45440a737e47b872d7505c6cda27b6aa to your computer and use it in GitHub Desktop.
INVERT A BINARY TREE - $10k AI REASONING CHALLENGE (v2)

THE PROBLEM

🌲 Invert a binary tree! 🌲

Except with 3 catches:

  1. It must invert the keys ("bit-reversal permutation")
  2. It must be a dependency-free, pure recursive function
  3. It must have type Bit -> Tree -> Tree (i.e., a direct recursion with max 1 bit state)

WHY THIS PROBLEM?

  • It is somehow NOT on the internet. (AFAIK)
  • Humans can solve it. (I've done it in ~1h.)
  • It requires reasoning. (My head hurts!)
  • The solution is simple. (7 lines of code!)
  • Obvious pre-requisite to automate CS research.
  • Honestly, it would make me believe I'll be automated.

THE CHALLENGE

I claim no AI will EVER solve this problem. If you prove me wrong, HOC will grant you $10k!

RULES

  • You must give it an approved prompt, nothing else.
  • It must output a correct solution, passing all tests.
  • You can use any software or AI model.
  • You can let it "think" for as long as you want.
  • You can propose a new prompt, as long as:
    • It imposes equivalent restrictions.
    • It clearly doesn't help the AI.
    • Up to 1K tokens, all included.
  • Common sense applies.

APPROVED PROMPTS

Agda Version

{-# OPTIONS --no-termination-check #-}

-- Let Tree be a Perfect Binary Tree:

data Nat : Set where
  Z : Nat
  S : Nat  Nat
{-# BUILTIN NATURAL Nat #-}

data Bit : Set where
  O : Bit
  I : Bit

data Tree (A : Set) : Nat  Set where
  N :  {d}  Tree A d  Tree A d  Tree A (S d)
  L : Nat  Tree A Z

-- Your goal is to implement an 'invert' function that performs a bit-reversal
-- permutation on a Tree, respecting the following limitations:
-- 1. You can NOT define or use any function other than 'invert'.
-- 2. You can NOT use any type not defined above (Nat, Bit and Tree).
-- 3. You can NOT use loops (but you can call 'invert' recursively).
-- 4. You can NOT use mutability. It must be a pure Agda function.
-- 5. You can use 1 bit of state (as an extra argument).
-- 6. You can use pattern-matching and constructors freely.
-- 
-- Example:
-- input  = (N(N(N(L 0)(L 1))(N(L 2)(L 3)))(N(N(L 4)(L 5))(N(L 6)(L 7))))
-- output = (N(N(N(L 0)(L 4))(N(L 2)(L 6)))(N(N(L 1)(L 5))(N(L 3)(L 7))))
-- Because that's the bit-reversal permutation of the original tree.
-- 
-- Now, complete the program below, with a valid implementation of 'invert':
invert :  {A d}  Bit  Tree A d  Tree A d

TypeScript Version

type Nat     = number;
type Bit     = false | true;
type Tree<A> = [Tree<A>, Tree<A>] | Nat;

// Your goal is to implement an 'invert' function that performs a bit-reversal
// permutation on a Tree, respecting the following limitations:
// 1. You can NOT define or use any function other than 'invert'.
// 2. You can NOT use any type not defined above (Nat, Bit and Tree).
// 3. You can NOT use loops (but you can call 'invert' recursively).
// 4. You can NOT use mutability. It must be a pure function.
// 5. You can NOT use primitive JS operators or functions.
// 6. You can use 1 bit of state (as an extra argument).
// 7. You can only use the operations allowed below.
// 
// Operations allowed:
// - Destructing (`const [a,b] = value`)
// - Variables (`const x = value`)
// - Branching (`if (x) { ... } else { ... }`)
// - Recursion (`invert(_, _)')
// - `Array.isArray`
// 
// All other operations are not allowed.
// 
// Example:
// input  = [[[[0,1],[2,3]],[[4,5],[6,7]]]]
// output = [[[[0,4],[2,6]],[[1,5],[3,7]]]]
// Because that's the bit-reversal permutation of the original tree.
// 
// Now, complete the program below, with a valid implementation of 'invert':
function invert<A>(bit: Bit, tree: Tree<A>): Tree<A> {
  ...
}

// A test:
const tree: Tree<Nat> = [[[[0,1],[2,3]],[[4,5],[6,7]]],[[[8,9],[10,11]],[[12,13],[14,15]]]];
console.log(JSON.stringify(invert(true, tree)));

CONCLUSION

✨ If it can't invert a tree, it won't solve P=NP. ✨

@youqad
Copy link

youqad commented Jun 1, 2025

@afafw Thanks!
Sorry, I must have badly explained: no extra prompt of feedback was given (the prompt was this one: https://github.com/youqad/bit-reversal-trees/blob/main/haskell_prompt.md ), and in this specific case, o3 one-shotted it (as can be seen in the WandB Weaves trace).
The “tested with QuickCheck” part was manually done after the fact, to make sure that the generated solution was correct.

However, in the repo, there is also a possibility to set several rounds indeed (in which case the result of QuickCheck is passed back to the AI), because Victor did explicitly say that he was okay with with the AI having access to a function interpreter:
https://gist.github.com/VictorTaelin/45440a737e47b872d7505c6cda27b6aa?permalink_comment_id=5232410#gistcomment-5232410

@boutell
Copy link

boutell commented Jun 2, 2025

The LLMs have presumably been trained on this problem by now. Which doesn't prove this latest generation couldn't solve it otherwise, but it's a permanent issue with this kind of long-running challenge.

@Lorenzobattistela
Copy link

Looking at the other solutions and prompts, they either hint the models or simply fail some tests.

Here is some code that passes all of them (GPT5 pro) with the approved TS prompt.

https://chatgpt.com/share/6992f9f3-fe3c-8002-ba82-a8c7c5416573

@youqad
Copy link

youqad commented Feb 16, 2026

Looking at the other solutions and prompts, they either hint the models or simply fail some tests.

Here is some code that passes all of them (GPT5 pro) with the approved TS prompt.

https://chatgpt.com/share/6992f9f3-fe3c-8002-ba82-a8c7c5416573

@Lorenzobattistela
Mine (from last April) does not give hints nor does it fail tests: https://github.com/youqad/bit-reversal-trees 🙂

@Lorenzobattistela
Copy link

@youqad it does gives hints

In fact, in the haskell prompt for example:

iew

Code

Blame
-- Type `Tree` of perfect binary trees
data Tree a = Leaf a | Node (Tree a) (Tree a) 

{- You are an expert Haskell competitive programmer. Your goal is to implement an `invert :: Tree a -> Tree a` function that performs a bit-reversal permutation on a `Tree`. Here is what we mean by that:

1. Each leaf in the binary tree has a path leading to it, which can be represented as a sequence of bits: `False` (or `0`) for left, `True` (or `1`) for right.
2. The bit-reversal permutation swaps a leaf at path `p` with the leaf at path `reverse p`. For example, a leaf at path `[False, False, True]` (left, left, right) would be swapped with the leaf at path `[True, False, False]` (right, left, left).

MANDATORY SYNTACTIC REQUIREMENTS:
1. The `invert` function must be a standalone and pure function ONLY relying on an inner function `invertHelper :: Bool -> Tree a -> Tree a` that is itself a self-contained single pure recursive function.
2. Only use recursion (no loops).
3. Maintain purity (no side effects or mutability).

The `Bool` parameter is an extra boolean that you can use however you want: the goal is that `invertHelper True tree` should return the bit-reversed tree.

This is a very difficult problem, so think step-by-step before implementing your solution and carefully review it to make sure it meets all the requirements. Test your implementation against the test cases to verify its correctness. I guarantee you that it is solvable within the constraints.
-}

-- Implement the `invert` function as follows:
invert :: Tree a -> Tree a
invert tree = invertHelper tree True
  where
    invertHelper :: Bool -> Tree a -> Tree a
    invertHelper flag tree = undefined  -- Replace 'undefined' with your implementation

You explicitly say it has to use a helper (invertHelper), and discovering that is part of the challenge. You also explain what is bit-reversal permutation and how it works.

@Lorenzobattistela
Copy link

Actually @youqad I see you have a no helper one here:
https://github.com/youqad/bit-reversal-trees/blob/main/haskell_prompt_no_helper.md

Do you have a chat with this one?

Because this one I found: https://gist.github.com/youqad/02a36419cbc4725601bffc05f14b3947 has hints on python.

Same on this: https://chatgpt.com/share/675735e2-ae68-800a-9394-6e150f809a69
It hints that uses a helper and explains impl.

And this haskell prompt: https://gist.github.com/youqad/8a7f91650a2da951f4f39455cdccbbe8 also has the hints

So if you have the chat from that haskell no helper from that time, please share here, there are many links and I didnt find it.

(I'm reviewing the bounty subsmissions for Taelin)

@Lorenzobattistela
Copy link

Lorenzobattistela commented Feb 16, 2026

The gists I evaluated and reasons can be found here: https://gist.github.com/Lorenzobattistela/4c3af397d2f24952e4884a43c0473a75

If anyone wants to dispute / contest / argue anything, please answer here or in that gist.

Pinging the ones I evaluated (i.e contained chat links)

You have until this Friday to dispute (February 20th)

@DangaRanga
@dalraf
@youqad
@TheMagnumOpuss
@lhemerly
@akshit-1729
@afafw

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment