The growing popularity of artificial intelligence (AI) has led many to wonder if this is the next tech boom that will end in six months.
However, recent benchmark tests show that Category ID It reveals how far GPT-4 has come and suggests it could be a game changer for the web3 ecosystem.
Debug testing of AI code
The data below shows some tests across available open-source Large Language Models (LLMs) similar to OpenAI’s ChatGPT-3.5 and GPT-4. Category ID tested We used the same sample of C+ code for each model and recorded the number of false alarms for errors and bugs identified.
LLaMa 65B (4-bit GPTQ) model: 1 false alarms in 15 good examples. Detects 0 of 13 bugs.
Baize 30B (8-bit) model: 0 false alarms in 15 good examples. Detects 1 of 13 bugs.
Galpaca 30B (8-bit) model: 0 false alarms in 15 good examples. Detects 1 of 13 bugs.
Koala 13B (8-bit) model: 0 false alarms in 15 good examples. Detects 0 of 13 bugs.
Vicuna 13B (8-bit) model: 2 false alarms in 15 good examples. Detects 1 of 13 bugs.
Vicuna 7B (FP16) model: 1 false alarms in 15 good examples. Detects 0 of 13 bugs.
GPT 3.5: 0 false alarms in 15 good examples. Detects 7 of 13 bugs.
GPT 4: 0 false alarms in 15 good examples. Detects 13 of 13 bugs.Open source LLM was only able to detect 3 out of 13 bugs in 6 models, but identified 4 false positives. On the other hand, GPT-3.5 caught 7 out of 13 bugs, and OpenAi’s latest offering, GPT-4, caught all 13 out of 13 bugs without false alarms.
Breakthroughs in bug detection have the potential to revolutionize the deployment of smart contracts on web3, apart from the myriad other web2 sectors that offer huge benefits. For example, web3 connects digital activities and assets with financial instruments, giving it the name “Internet of Value”. Therefore, it is very important that all code running in the smart contracts that power web3 is free of bugs and vulnerabilities. A single entry for a malicious attacker can lose billions of dollars in an instant.
GPT-4 and AutoGPT
GPT-4’s impressive results justify the current hype. Additionally, the power of AI is within reach to help ensure the security and stability of the evolving web3 ecosystem.
Applications such as AutoGPT have emerged to allow OpenAI to create other AI agents to delegate work tasks. It also uses Pinecone for vector indexing, allowing access to both long-term and short-term memory storage to address GPT-4 token limitations. Several times over the last week, the app has been trending on Twitter all over the world from people who have launched their own AI agent armies around the world.
If AutoGPT can be used as a benchmark to develop similar or forked applications to continuously monitor, detect bugs, and suggest solutions for upgradeable smart contract code. there is. These edits may be manually approved by the developer or his DAO, ensuring that there is a “human in the loop” approving deployment of the code.
You can also create similar workflows to deploy smart contracts through bug reviews and simulated transactions.
reality check?
However, technical limitations need to be overcome before AI-managed smart contracts can be deployed in production. Catid’s results reveal a limited scope of testing, but a focus on short code where GPT-4 excels.
In the real world, applications contain multiple files of complex code with myriad dependencies that quickly exceed the limits of GPT-4. Unfortunately, this means that his GPT-4’s performance in real-world conditions may not be as impressive as the tests suggest.
However, it is clear that the question is no longer whether the perfect AI code writer/debugger is feasible. The question is what ethical, regulatory and agency issues arise. Additionally, applications like AutoGPT are very close to being able to autonomously manage their codebases with vectors and additional AI agents. The limitations are mainly in the robustness and scalability of the application, which can get stuck in a loop.
the game is changing
GPT-4 is only a month old, but there are already a wealth of new public AI projects, such as AutoGPT and Elon Musk’s X.AI, reimagining future conversations about the technology.
The crypto industry seems poised to harness the power of models like GPT-4 as smart contracts that offer ideal use cases for creating truly autonomous and decentralized financial instruments. is.
How long will it take to see the first truly autonomous DAO with no humans in the loop?
Post Here’s why GPT-4 is better than GPT3.5: LLM in code debugging first appeared in CryptoSlate.
