Coding assistance AI tools

Danny Briskin, Quality Engineering Practice Manager

Introduction

In recent years, artificial intelligence has significantly transformed the landscape of software development by introducing AI coding assistance tools. These tools, powered by advanced machine learning algorithms, are designed to enhance productivity, reduce errors, and streamline the coding process. By providing real-time suggestions, auto-completions, and intelligent code analysis, AI coding assistants are revolutionizing the way developers write and optimize code. This technology not only accelerates the development process but also empowers developers to focus on more complex problem-solving tasks, thereby fostering innovation and efficiency.

Are those tools really that powerful? Let’s figure that out.

AI powered tools

There are plenty of tools that can be used for software development process, starting from idea generation to code generation and testing. Here is a list of most know ones

• Tools for ideas generation and concepts creation
  • ChatGPT - A universal assistant for generating ideas and answering complex queries.
  • GlueCharm - A tool for creating user stories and managing product backlogs.
  • Frase - A content generation tool for websites.
  • Miro - A collaborative platform for product diagramming and prototyping.
• Tools for coding assistance
  • Copilot - A code completion tool that suggests and generates code based on the context.
  • ChatGPT models - Can assist in code generation and problem-solving.
  • Gemini - A text-generation tool.
  • Vertex AI - A tool for both text and code generation.
  • Gemini Code Assist - A text-generation tool for writing and generating code.
  • Llama - Meta’s code and text generation model.
  • Tabnine - A robust AI code completion tool.
  • Codeium - Specializes in both code and text generation, offering advanced capabilities for developers.

In this article I will concentrate on coding assistance tools only.

The Pros of AI tools in Software Development

AI coding tools have become an integral part of modern software development, offering a range of benefits:

• Affordability: Many AI tools provide a trial period or even a free tier, making them accessible to a broad audience.
• Deployment Options: These tools often offer cloud-based solutions and on-premises versions to suit different organizational needs.
• Ease of Use: Installation is typically straightforward, and most tools support plugins for popular IDEs, integrating seamlessly into existing workflows.
• Contextual Suggestions: AI tools utilize predefined models and analyze your codebase, including comments, to provide context-aware suggestions.
• Security: Many AI tools are tested for security compliance and have relevant certifications.
• Specific Use Cases:
  • Function Documentation and Comments: AI tools are extremely useful for generating javadoc, docstrings, and in-code comments, especially for non-native English speakers.
  • Code Predictions: These tools are helpful for generating boilerplate code, but when the code is supposed to be more sophisticated, the usefulness of tools becomes questionable.
  • Code Completion: AI code completion is useful but built-in IDE code completion tools are currently more robust and faster.
  • Git Commit Messages: Some tools can analyze changes between commits, though they are not very robust.
  • Code Explanation: AI tools provide valuable insights for junior developers, making them very useful in this aspect.
  • Unit Test Generation: Many tools excel in generating unit tests, offering great value there.
  • Code refactoring: - tools are quite useful, but developer needs to put a lot of efforts to explain what is needed and the result is questionable in most cases.

Code Assistance with Codeium – Use Case Analysis

AI-powered tools such as Codeium have proven to be effective in generating code documentation, ranging from simple to more complex cases. For example, the following Python docstring was entirely generated by AI:

"""
Sends a REST request to the specified endpoint with the given parameters.

Args:
    endpoint_qualifier (str): The identifier of the endpoint to send the request to.
    endpoints_config (dict[str, Any]): A dictionary containing the configuration for all endpoints.
    replacements (dict[str, dict[str, str]], optional): A dictionary containing replacements for the payload and path of the request.
    **kwargs: Additional keyword arguments to customize the request.

Returns:
    Response: The response object containing the result of the request.

Raises:
    ValueError: If the endpoint qualifier is invalid or if the authentication or method is not specified for the endpoint.
    Exception: If the expected status code does not match the actual status code of the response.
"""

On the other hand, this docstring output

although highly useful, still requires human review for completeness and correctness.

Code Predictions

The code prediction functionality provided by Codeium is generally effective, particularly in scenarios involving repetitive tasks like REST requests. It efficiently predicts the next line of code and offers useful suggestions.

However, if a prediction is declined and replaced with a manually written line, the tool will still attempt to predict the same suggestion for the subsequent line, which can become frustrating for the developer after multiple occurrences.

Code Completion

The code completion feature is closely related to prediction but often lacks in variability and performance. For example, when a user starts typing, the AI might suggest a single line of completion, as seen in the grey suggestion below: In contrast, the built-in IDE completion tools typically provide multiple options, as seen in the popup shown in the same scenario. This demonstrates that while AI completion can be helpful, its current implementation still trails behind traditional code completion systems in terms of flexibility and speed.

Git Commit Messages

The AI’s ability to assist in generating Git commit messages is somewhat limited. When analyzing changes in the deletion of a class field (a mapped database table column, for instance), the suggested commit messages remain very basic, often failing to capture the full context of the modification. For example: When the change was in deletion of class field (mapped database table column): The proposed commit messages were (white was human printed, grey – AI propositions): Or: As seen in this instance, the AI lacks the sophistication to create insightful commit messages and typically defaults to generic or overly simplistic suggestions.

Code Explanation

This feature is highly beneficial, especially for junior engineers. The AI can identify third-party libraries and explaining their methods based on a minimal codebase. This ability to provide clear, context-aware explanations can significantly accelerate the learning curve for less experienced developers.

Unit Test Generation

AI-generated unit tests can be valuable as a starting point. The AI is proficient in producing a list of possible test cases and generating basic code structures for unit tests. For instance, when provided with a function to test, the AI was able to suggest relevant tests and create functional test cases.

While the tests may need further customization and refinement, the foundation provided by AI can save developers time and ensure greater test coverage.

Code Refactoring

For simple and well-understood scenarios, AI-driven code refactoring works efficiently. For instance, when refactoring the following function: A refactoring with: resulted in the next code:

A parameter was added, the correct field name was found in the Account_Balancing class, correct function (where) was called.

But, if the scenario becomes a bit more complicated, the AI fails. Trying to add a JWT token type authentication functionality along with existing Bearer Authentication: Here you are (that’s right, it’s a carbon copy):

Even a straightforward call to an existing function from the same file results in complete guesswork from the AI, as if it is uncertain about the expected outcome. For example:

Using the rest_request function from this file, send a POST request to http://localhost:8080/endpoint with payload {"onekey": "twovalue"}

The answer was: The function to call looks like this: As a result, the parameter usage was entirely incorrect.

The Cons of AI Tool in Software Development

Despite their advantages, AI coding tools have several drawbacks:

• Immature Plugins: IDE plugins need improvement to become more reliable and feature-rich.
• Performance Issues: The analysis required for suggestions can lead to slow performance, affecting productivity.
• Data Privacy Concerns: There is a risk of data leaks, as user inputs may be used for model training by AI tool creators.
• Lack of Open Source: Most tools are not open source, raising concerns about potential information leaks, even with on-premises solutions.
• Training Data Quality: The models are trained on unknown datasets, which may not adhere to best coding practices.
• Limited of Precise Tuning: While AI tools can use your codebase for suggestions, they often focus on the current file rather than the entire project.
• Annoying Behavior: AI tools may repeatedly suggest inappropriate code, causing frustration for developers.

General Concerns about AI Tools

• Reliability: AI tools can generate erroneous or unsafe code, necessitating thorough code reviews and undermining the idea of “machine-generated code.” Complex tools may create code that contradicts programming principles.
• Code Support and Maintenance: AI-generated code may be overcomplicated, use uncommon approaches, and vary among developers, making collaboration challenging. Refactoring AI-generated code can be difficult.
• Continuous Improvement: AI is trained (at least it is supposed to be) on syntactically correct elements, but this doesn’t guarantee optimal performance or maintainability.
• Loss of Control: Developers may feel disconnected from the code, leading to their skill deterioration and reduced ability to handle AI errors.
• Habits and Motor Memory: Experienced developers rely on motor memory for IDE tasks. AI tools disrupt this pattern, requiring more time for decision-making and returning to the original workflow.
• Incorrect Suggestions: AI tools may offer irrelevant code suggestions that appear correct, leading to potential coding errors that regular code completion tools would avoid.

Summary

AI coding assistance tools have revolutionized software development by enhancing productivity, reducing errors, and streamlining the coding process. These tools provide real-time suggestions and intelligent code analysis, allowing developers to focus on complex problem-solving. AI tools offer several benefits, including affordability, ease of use, and contextual suggestions. They excel in specific tasks such as function documentation, code predictions, unit test generation, and code explanations, particularly aiding non-native English speakers and junior developers.

However, AI tools also have significant drawbacks. They can suffer from performance issues, data privacy concerns, and immature plugins. Most tools are not open source, leading to potential information leaks and questionable training data quality. AI-generated code can be unreliable, challenging to maintain, and often requires thorough review. Developers may feel a loss of control over their code, and AI tools can disrupt their workflow and motor memory. Additionally, incorrect suggestions can lead to coding errors that are not easily caught.

Overall, while AI tools can greatly assist in software development, their limitations and risks must be carefully managed to ensure reliable and efficient coding practices.