Quick update on the AI assistant.

V2.0 is now out and is adaptable across any PDF I have thrown at it. The books I use have been purchased from Nostarch press due to the fact that Nostarch press sells digital PDF’s of many technical books. This has been incredibly helpful. Originally I had bought a few physical copies of books from Nostarch press, so I sent the Nostarch press support an email inquiring about being able to obtain the digital copies of those books, mainly the PDF versions. Nostarch got back to me with codes for digital downloads a few for $5/book and I think six additional books at $10/book. This made things much cheaper for me, so thank you Nostarch press!

The V2.0 AI assistant has been fine-tuned and tested and approved (by me), but can also be changed by anyone as I have made the code opensource (https://github.com/TheDumbEngineer/AI_Assistants/blob/main/ReverseEngineeringAssistantV2_0_RAG_PDF.ipynb). I created my own prompt that is available from LangSmith prompt hub (https://smith.langchain.com/hub/byteberzerker/reverse_helper). Please feel free to apply some magic prompt engineering to my prompt.

Ok, so I really need to change the name of my prompt and my AI assistant, it’s more of a general tool now. Let me know if you have any interesting names, or maybe i’ll ask AI to generate a few for me.

Stay Tuned!

Good news! version 1.0 of the AI reverse engineering assistant is complete. The assistant was developed in Google Colab Pro using Langchain and OpenAI. Google Colab Pro and OpenAI API both cost money, though are relatively inexpensive in terms of what you get. Google Colab Pro you get a much better runtime environment (use a better GPU) for $9.99/Month and the OpenAI API is inexpensive, I put $10 on my account and I have $9.97 left over the few days of testing, I’ve made well over 10 API calls, can’t recall the exact number (I am the dumb engineer, after all…sigh).

The model utilizes what’s called a Retrieval-Augmented Generation or RAG for short, this technique enhances the model with your own provided knowledge. I’ll let Nvidia explain it better here: RAG Link. Utilizing LangChain and Python for the programming made the overall process simple, thank you LangChain!

Step-by-step overview on how the RAG technique works with LangChain:

1. Load the data with a document loader, I decided to use unstructured pdf data, LangChain has many different loaders.

2. Split the document into smaller chunks which allows the model to process the data better.

3. Store the data in a vector database, I used chroma because it is opensource. There are many other ways to do this, a quick google search of vector databases for AI using LangChain should generate some interesting results.

4. Retrieval and Generation, this is the bread and butter we take a user input (question) and then the retreiver searches for documents relevant to the question, then passes the retrieved docs and question to the model for the magic.

LangChain RAG Docs Example

GitHub Links to my code:

https://github.com/TheDumbEngineer/AI_Assistants/tree/main

Stay Tuned!

The initial lab has been setup, and I wanted to give you the information on what I will be using to create and test the AI reverse engineering assistant.

1. Dev Platform: Google Colab Pro+, Visual Code, and GitHub.

2. Programming: Python, Lang-Chain, and OpenAI.

Google Colab Pro+ is a paid subscription, the cost is $9.99 per month. I chose to purchase Pro+ because of the ability to choose a strong GPU to run AI programs, which is good in my case due to hardware limitations.

The rest of the lab is completely free, and note that colab is free to use as well, but if you want the option to run a strong GPU you will need to purchase the subscription.

Standby!