How to debug codes generated by ChatGPT

Debugging code generated by ChatGPT by pasting the error into ChatGPT for it to resolve its own problem, most of the time, is a very bad choice for me. So, most of the time, I have to debug the codes by myself. So, here are some tips, based mostly on the fact that ChatGPT uses Generative Large Language Model to generate code.

1. Be Aware of Hallucinated Code

ChatGPT can generate:

• Non-existent Functions: E.g., suggesting pd.some_nonexistent_function().
• Exist function but wrong sub-module: E.g., ´from sktime.forecasting.compose import make_pipeline´ while it should be ´from sktime.pipeline import make_pipeline´
• Incorrect Syntax: Mixing features from different libraries or creating imaginary parameters.

Solution:

• Double-check any unfamiliar functions or parameters in the official documentation of the package
• Test the code as soon as it’s generated to catch these issues early.

2. Library Mix-Ups

ChatGPT may confuse similar libraries or frameworks due to their overlapping functionality. For example:

• Using plt.bar() (Matplotlib) but applying Seaborn-style parameters.
• Suggesting TensorFlow functions but using PyTorch syntax.

Solution:

• Specify the exact library to use in your prompt (e.g., “Use Matplotlib for plotting, not Seaborn”).
• Look for consistency in the imports and function calls.

3. Outdated or Deprecated Features

Generative models like ChatGPT might reference:

• Deprecated Functions: E.g., suggesting pandas.Panel() (removed in recent versions).
• Old APIs: Using an older version of a library when your environment uses the latest.

Solution:

• Check for version mismatches by comparing the suggested functions with the library’s changelogs.
• Use pip show <library> or conda list to identify the installed version of a library.

4. Overly Simplistic Solutions

ChatGPT often generates code for the most common use case, ignoring edge cases or performance optimizations.

Example:
Sorting a large dataset in a naive way without considering efficiency.

data.sort()  # May not handle very large datasets efficiently.

Solution:

• Review the code for scalability and edge cases. You can also ask ChatGPT to refine its code for scalability,….
• Optimize the logic where necessary (e.g., using parallel processing for large datasets).

5. Placeholder Variables and Logic

ChatGPT often generates placeholder-like variables or overly generic solutions.

Example:

for item in list:  # Generic variable names like `list` may shadow built-ins.
    process(item)

Solution:

• Replace placeholders (list, item, etc.) with meaningful, descriptive names.
• Revisit the logic to ensure it aligns with your specific requirements.

6. Assumptions in Data Structures

ChatGPT may assume:

• Fixed Input Formats: Generating code that assumes structured data without handling irregularities.
• Missing Validation: Lack of error handling for inputs.

Example:

def divide(a, b):
    return a / b  # No check for division by zero.

Solution:

• Add input validation and error handling (again, you can also ask ChatGPT to add this detail for you).

def divide(a, b):
    if b == 0:
        raise ValueError("Division by zero is not allowed.")
    return a / b

7. Incorrect Contextual Understanding

ChatGPT might misunderstand your intended use case and generate unrelated or partially relevant code.

Solution:

• Provide a clear, detailed prompt describing your requirements.
• If the algorithm consists of many small steps, guide chatGPT to code a few steps each time, to make sure that the current steps are correct before moving on to the next steps.
• Review the generated code’s alignment with the context and adjust the logic if needed.

8. Over-Complexity in Logic

ChatGPT might produce unnecessarily complex code for simple tasks.

Example:
Calculating the sum of a list:

total = 0
for num in numbers:
    total += num

Instead of using:

total = sum(numbers)

Solution:

• Simplify logic wherever possible.
• Replace loops with built-in functions or more Pythonic alternatives.

9. Missing Dependencies

ChatGPT might reference functions or libraries without importing or installing them.

Example:

plot_data(data)  # Without importing Matplotlib or another plotting library.

Solution:

• Add the missing imports.
• Install the required dependencies using pip or conda commands.

11. Static Values in Dynamic Contexts

ChatGPT may hard-code values instead of making them dynamic. While this may not cause trouble at the first run, it may cause trouble when you change some parameters in your codes.

Example:

n = 5
for i in range(5):  # Hardcoded loop range.
    print(i)

Solution:

• Replace static values with variables or user inputs.

n = int(input("Enter a number: "))
for i in range(n):
    print(i)

12. Ask ChatGPT for Clarifications

If something seems unclear:

• Ask for Explanations: Request explanations for specific lines or blocks.
• Refine the Prompt: Use follow-up questions to refine the generated code.

By anticipating these quirks and following structured debugging steps, you can effectively use and improve ChatGPT-generated code for your projects.