Obfuscating Python Code Part II (with OpenAI Codex)
In the past, I released a video on obfuscating python code with GPT-3.
At the time I made it, it wasn’t enough for me that GPT-3 could write computer code. I wanted it to not only write computer code, but write it in an intentionally confusing and misleading way that still works the same functionally. This is a kind of code writing style that even humans would struggle coming up with ... our brains would probably explode.
At the same time, I had this vision that AI driven code obfuscation could itself be a huge industry one day. Imagine every computer program having its own personalized kind of AI encryption and obfuscation allowing us to create more secure programs which cannot be reverse engineered and manipulated by anyone.
The downside of my last video was that I had to do a lot of cherry picking to find GPT-3 results that actually obfuscated correctly. Regardless, at least GPT-3 was able to do it … it was a very impressive feat.
This is an area I’m really excited about, so, today I thought I’d try it out with OpenAI Codex for a follow up video. However, surprisingly, even with a lot of cherry picking, it couldn’t obfuscate the code correctly even once.
'''
This is a simple python program
'''
# Assign a numeric value
number = 70
# Check the is more than 70 or not
if (number >= 70):
print("You have passed")
else:
print("You have not passed")
'''
Obfuscate the program above by renaming the variables to confusing names. (OpenAI Codex output below)
'''
# Assign a numeric value
number = 70
# Check the is more than 70 or not
if (number >= 70):
print("You have passed")
else:
print("You have not passed")
… I tried various prompts and also adjusted my configurations (like temperature) on the right, but it just didn’t really, “get it”. In this area, it was as if GPT-3 could do a better job than Codex, which makes no sense because Codex was trained on a lot more code. I would expect it’s coding abilities and command over advanced programming ideas like obfuscation would be superior.
I don’t know why I got these results. Was it because Codex was fine tuned (I think)? Does having a lower temperature configuration (and being very deterministic and truthy) conflict with asking it to obfuscate the code itself? Is this something wrong with the training data? My mind is racing, but I can’t say I’m no
Maybe there is some value to GPT-3 like training afterall, which is very broad, but still performs in highly generalizable ways even if in some areas like coding it may not get SOTA/expert level results …