Hi Folks,
Good news! version 1.0 of the AI reverse engineering assistant is complete. The assistant was developed in Google Colab Pro using Langchain and OpenAI. Google Colab Pro and OpenAI API both cost money, though are relatively inexpensive in terms of what you get. Google Colab Pro you get a much better runtime environment (use a better GPU) for $9.99/Month and the OpenAI API is inexpensive, I put $10 on my account and I have $9.97 left over the few days of testing, I’ve made well over 10 API calls, can’t recall the exact number (I am the dumb engineer, after all…sigh).
The model utilizes what’s called a Retrieval-Augmented Generation or RAG for short, this technique enhances the model with your own provided knowledge. I’ll let Nvidia explain it better here: RAG Link. Utilizing LangChain and Python for the programming made the overall process simple, thank you LangChain!
Step-by-step overview on how the RAG technique works with LangChain:
Load the data with a document loader, I decided to use unstructured pdf data, LangChain has many different loaders.
Split the document into smaller chunks which allows the model to process the data better.
Store the data in a vector database, I used chroma because it is opensource. There are many other ways to do this, a quick google search of vector databases for AI using LangChain should generate some interesting results.
Retrieval and Generation, this is the bread and butter we take a user input (question) and then the retreiver searches for documents relevant to the question, then passes the retrieved docs and question to the model for the magic.
GitHub Links to my code:
https://github.com/TheDumbEngineer/AI_Assistants/tree/main
Stay Tuned!