Everybody’s heard of ChatGPT at this stage. It’s an AI-powered chatbot that makes use of natural language processing models to produce answers to complicated questions. Those answers may not necessarily always be accurate, but they tend to be detailed enough that you can fact-check the answers that it gives you fairly easily. What if you were to take ChatGPT and try to write code with it?
As it turns out, that’s something you can do with varying degrees of success, depending on the languages you want to use and what you want to achieve with them. I put it to the test to see how well it would fare in different programming languages to complete different tasks. I also won’t be making any changes to the code that ChatGPT produces, I will only guide it to create working code if the code that it creates doesn’t work.
I used three languages that I am already familiar with to put ChatGPT to the test: Python, C, and Java. The results were particularly surprising as it struggled a little bit with Python because of the abstract nature of what I was asking it to do, but it handled C and basic Java with ease. I have also created a GitHub repository with all of the code here accessible so that you can try it out yourself.
Automatic page checking in Python
Python is my favorite programming language to work in, and that’s thanks to the power that it has and its simplicity of use. I have written many tools in Python in the past to automate parts of my workflow, and my Bachelor’s thesis in computer science was even done entirely through Python. I asked ChatGPT to write a program that could scan a webpage for changes on a periodic timed basis, and it mostly did the trick.
The issue with the above code that it gives me is that it doesn’t simulate a real browser. Websites check for a parameter in your browser called a “user agent,” and this defines what kind of browser you’re using or if you’re a bot. Many websites won’t accept connections that don’t define a user agent, and I ran into this when I pointed it at XDA. I asked it to add custom headers to the request to simulate a real browser, and it complied.
The code given above works for static websites, but the problem is that much of the content that you read online is dynamic. If a page returns the current time, for example, somewhere on the page, then the code above will always say that the page has changed every time it refreshes. When I pointed this out to ChatGPT, it got confused and gave me an entirely incorrect solution to the problem.
The above is an incorrect explanation of how hashes work. The point of a hash is that a minor change to the input will result in a major difference in the hash, in that two hashes are basically incomparable. When I told it that it was an incorrect assertion to suggest that hashes were the best way to compare and scan for changes, it replied with an apology and gave me something that did work.
The only problem that I’ve discovered with the above code is that for larger websites, it’s computationally expensive as it uses natural language processing to compare and calculate the difference between them. Still, a 5% threshold in changed content is good enough for most websites if you’re looking to check for updates, and you can adjust the threshold if needs be. It took a while, but it got there in the end.
There are a lot of reasons you may want a web scraper like one of these, and while there are tools out there that can do it, it’s cool to build your own. You can use them for package tracking or, in my case, scanning for new properties that go up for rent on local property sites.
Password generator in C
With security being all the rage recently, it’s important that you use secure passwords and also use two-factor authentication. If you struggle with the secure password part, it’s pretty easy to write up a password generator in C. ChatGPT handled it pretty easily, and the only problem I discovered was that when I asked it to create a password generator, it didn’t import the String C library. I told it that, and it added it to the imports at the start.
I increased the MAX_LENGTH global variable to 32 characters instead of 16, and it generated the following password for me: aW3H0E(&FPQvG8B@4*()+4yRKTKB#U0O. Safe to say that it works!
Calculator in Java
A pretty basic program that many people have experience with in Java is the creation of a calculator. It’s one of the earliest challenges that computer scientists will face in their programming career, no matter what language they take up first. I asked ChatGPT to write a calculator in Java that made use of at least two classes, and it worked perfectly on the first attempt.
This is the calculator class:
This is the main class:
Creating both of these classes in my IDE, IntelliJ, and executing the main class results in a perfectly working calculator program that takes in two numbers and an operator for addition, subtraction, multiplication, or division.
ChatGPT is decent at programming
While I wouldn’t rely on ChatGPT to do everything for you, it’s clear that it can do a pretty good job at creating programs and taking critiques into account when there are problems. I was surprised to see it suggest a hashing algorithm as a way to discern between minor changes in a webpage, but aside from that, it didn’t struggle with the more basic tasks whatsoever. Even in the case of the Python program, it had the right idea from the very beginning and, on more static pages, would have worked perfectly.
Of course, ChatGPT isn’t exactly going to be replacing programmers anytime soon, but it’s on its way to being a very good tool in the workforce. GitHub’s Copilot, for example, makes use of GPT technology to help people write code, and it’s only going to get more advanced as time goes on. Even if you’re a beginner programmer, this kind of tool could potentially help you learn if you could trust it fully, but sometimes it makes weird mistakes, as we’ve seen.
Of course, the examples given here are very basic, and most programmers would be able to write them fairly easily, but that isn’t the point. It’s scarily good at building solutions to simple tasks and seems to handle more complex tasks decently well, too. We’re excited to see where tools like these go in the future and help people improve their skills.