sentient ai

[or] over-hyped regression model

2023/04/09

Tags: code ai machine learning neural networks

I recently came across this paper by the team in Microsoft experiment with the ChatGPT-4 model. The authors went through putting the model through a series of tests: from simple coding exercises, to philosophical discourses, mathematical problems etc. The results are quite impressive, I’ll be honest. Even though I have had no access to the GPT-4 model so far, I tried running some tests of my own with the GPT-3 model both on the OpenAI ChatGPT interface, and the Bing chat to see for myself just how well can the model perform on tasks that I would consider useful for my own specific needs.

To set the goals straight, I decided to give each model the same prompt, and use the code they produced as-is. If I saw that the model was close to what I needed, I would either refine my prompt and give it a chance to fix the code, or I would just copy-paste the code and fix it myself. The goal was to converge to the desired solution faster, than it would otherwise take me to write the code for myself. Below I will only be showing the final result. I will also grade each solution based on my very biased and subjective opinion. Without further ado, here are the results, and my final thoughts.

TaskModelGrade
Brownian motion (python)ChatGPT
Bing Chat
me--
Perlin noise (python)ChatGPT
Bing Chat
"me"--
1D diffusion solver (c++)ChatGPT
Bing Chat
Extract white regions (ffmpeg)ChatGPT
Bing Chat
me

1. Modeling and plotting brownian motion

Using python generate trajectories of N particles over time that do random walk. Each timestep particles moves a fixed distance at an arbitrary direction. The particles start uniformly distributed inside a circle. Prefer numpy operations over for loops. Plot the trajectories of the particles.

Results

The code ChatGPT produced is quite readable, and I personally like the comments that it left. I also admire the fact that it figured out it needs to take a sqrt from random numbers for r to distribute particles uniformly in the circle (otherwise, particles would have preference for r=0). Unfortunately, however, the code doesn’t really do what I asked. If you look closely, it generates a single directions vector for each particle that does not evolve in time, so particles end up moving in straight lines. I tried refining my prompt several times, but each time it would either produce the same code, or add bugs to the otherwise working code.

Surprisingly, Bing did a better job in that it generated the theta values on-the-fly, and actually produced the random walk it was asked for. Like ChatGPT, it figured out it needs to take a sqrt from random numbers for r to distribute particles uniformly in the circle. The plotting part, however, is quite messy, it did a scatter plot of each particle at every timestep, without any distinguishing feature between individual particles, so it’s quite hard to follow the trajectories.

Chat GPT-3

view the code
# Define parameters
N = 100  # number of particles
D = 2  # number of dimensions
steps = 100  # number of time steps
step_size = 0.1  # fixed distance particles move at each step
radius = 1.0  # radius of the circle particles start in

# Generate initial positions
theta = 2 * np.pi * np.random.rand(N)
r = radius * np.sqrt(np.random.rand(N))
x = r * np.cos(theta)
y = r * np.sin(theta)
pos = np.column_stack((x, y))

# Generate random directions for each particle
angles = 2 * np.pi * np.random.rand(N)
directions = np.column_stack((np.cos(angles), np.sin(angles)))

# Generate random walk trajectories
traj = np.zeros((N, D, steps + 1))
traj[:, :, 0] = pos
for i in range(steps):
    pos += step_size * directions
    traj[:, :, i + 1] = pos

# Plot trajectories
fig, ax = plt.subplots()
ax.set_xlim(-radius, radius)
ax.set_ylim(-radius, radius)
for i in range(N):
    ax.plot(traj[i, 0, :], traj[i, 1, :])
plt.show()

brownian motion chatgpt

Bing Chat

view the code
# Bing chat [1 prompt]

import numpy as np
import matplotlib.pyplot as plt

N = 100  # number of particles
T = 100  # number of timesteps
R = 5  # radius of circle

# initialize positions
theta = np.random.uniform(0, 2 * np.pi, N)
r = R * np.sqrt(np.random.uniform(0, 1, N))
x = r * np.cos(theta)
y = r * np.sin(theta)
positions = np.stack([x, y], axis=1)

# plot initial positions
plt.scatter(x, y)
plt.xlim(-R, R)
plt.ylim(-R, R)

# random walk
for t in range(T):
    # generate random directions
    theta = np.random.uniform(0, 2 * np.pi, N)
    # update positions
    positions += np.stack([np.cos(theta), np.sin(theta)], axis=1)
    # plot positions
    plt.scatter(positions[:, 0], positions[:, 1])
    plt.xlim(-R - np.sqrt(T), R + np.sqrt(T))
    plt.ylim(-R - np.sqrt(T), R + np.sqrt(T))

plt.show()

brownian motion bing

My solution

Both ChatGPT and Bing “integrated” the trajectories with an explicit for loop, which is not ideal for performance. In my solution, I generate the random directions for each particle at each timestep, and then use numpy broadcasting to update the positions of all particles at once, which is much faster as numpy operations are vectorized.

view the code
N = 1000
Nt = 100

phis = np.random.uniform(0, 2 * np.pi, (N, Nt))
rs = np.sqrt(np.random.random(N))
thetas = np.random.uniform(0, 2 * np.pi, N)

f = lambda proj: np.tile(rs * proj(thetas), (Nt, 1)).T + np.cumsum(proj(phis), axis=1)
xs = f(np.cos)
ys = f(np.sin)

plt.plot(xs.T, ys.T, "k", alpha=0.1, lw=0.5)
plt.gca().set_aspect("equal")

brownian motion me


2. Producing a 2D Perlin noise

Generate a 2D perlin noise of a given size using python (numpy). Prefer numpy operations over for loops. Plot the result for 512 x 512 resolution using matplotlib.

Results

ChatGPT didn’t really go anywhere after about 5 prompts, where I tried feeding it with the error the code was producing. The mismatched shapes, which is the most annoying thing to debug, was just too heavy for it to handle, and for me to try and ressurect. Unfortunately, it’s a hard fail for ChatGPT.

Bing Chat simply copied the solution from this stackoverflow thread which was indeed the correct solution, and technically it had all the rights to do so. Despite the fact that it was a bit too easy, and I was hoping for a bit more creativity, I have to give Bing Chat a pass on this one.

Chat GPT-3

view the code
def perlin_noise_2d(shape, res):
    def surflet(grid_x, grid_y, x, y):
        dist_x, dist_y = abs(x - grid_x), abs(y - grid_y)
        poly_x = 1 - 6 * dist_x**5 + 15 * dist_x**4 - 10 * dist_x**3
        poly_y = 1 - 6 * dist_y**5 + 15 * dist_y**4 - 10 * dist_y**3
        grad = grid[(grid_x % shape[0]), (grid_y % shape[1])]
        dot_prod = dist_x * grad[0] + dist_y * grad[1]
        return dot_prod * poly_x * poly_y

    def interp(x, func):
        return lambda idx: func(idx) * (1 - x % 1) + func(idx + 1) * (x % 1)

    # Generate a grid of vectors
    grid = np.random.normal(size=(shape[0] + 1, shape[1] + 1, 2))
    # Generate a coordinate grid of shape res
    coords = np.mgrid[0 : shape[0] : res[0] * 1j, 0 : shape[1] : res[1] * 1j].astype(
        np.int32
    )
    # Generate an empty noise image
    noise = np.zeros(shape)

    for i in range(res[0]):
        for j in range(res[1]):
            # Calculate the coordinates of the 4 surrounding grid points
            x_min, x_max = coords[0, i, j], coords[0, i + 1, j]
            y_min, y_max = coords[1, i, j], coords[1, i, j + 1]
            # Calculate the contributions from the 4 surrounding surflets
            s = (
                surflet(x_min, y_min, *coords[:, i, j])
                + surflet(x_max, y_min, *coords[:, i + 1, j])
                + surflet(x_min, y_max, *coords[:, i, j + 1])
                + surflet(x_max, y_max, *coords[:, i + 1, j + 1])
            )
            # Interpolate between those values based on the location within the square
            noise[x_min:x_max, y_min:y_max] += interp(
                coords[0, i, j] / res[0], interp(coords[1, i, j] / res[1], s)
            )

    return noise


# Generate the noise image
noise = np.zeros((512, 512))
for i in range(4):
    noise += perlin_noise_2d((512, 512), res=(16 * 2**i, 16 * 2**i)) / 2**i

# Plot the noise image
plt.imshow(noise, cmap="gray")
plt.show()
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[12], line 49
     47 noise = np.zeros((512, 512))
     48 for i in range(4):
---> 49     noise += perlin_noise_2d((512, 512), res=(16 * 2**i, 16 * 2**i)) / 2**i
     51 # Plot the noise image
     52 plt.imshow(noise, cmap="gray")

Cell In[12], line 33, in perlin_noise_2d(shape, res)
     30 y_min, y_max = coords[1, i, j], coords[1, i, j + 1]
     31 # Calculate the contributions from the 4 surrounding surflets
     32 s = (
---> 33     surflet(x_min, y_min, *coords[:, i, j])
     34     + surflet(x_max, y_min, *coords[:, i + 1, j])
     35     + surflet(x_min, y_max, *coords[:, i, j + 1])
     36     + surflet(x_max, y_max, *coords[:, i + 1, j + 1])
     37 )
     38 # Interpolate between those values based on the location within the square
     39 noise[x_min:x_max, y_min:y_max] += interp(
     40     coords[0, i, j] / res[0], interp(coords[1, i, j] / res[1], s)
     41 )

Cell In[12], line 12, in perlin_noise_2d..surflet(grid_x, grid_y, x, y)
     10 poly_x = 1 - 6 * dist_x**5 + 15 * dist_x**4 - 10 * dist_x**3
...
---> 12 grad = grid[(grid_x % shape[0]), (grid_y % shape[1])]
     13 dot_prod = dist_x * grad[0] + dist_y * grad[1]
     14 return dot_prod * poly_x * poly_y

IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

Bing Chat

view the code
def perlin(x, y, seed=0):
    # permutation table
    np.random.seed(seed)
    p = np.arange(256, dtype=int)
    np.random.shuffle(p)
    p = np.stack([p, p]).flatten()
    # coordinates of the top-left
    xi = x.astype(int)
    yi = y.astype(int)
    # internal coordinates
    xf = x - xi
    yf = y - yi
    # fade factors
    u = fade(xf)
    v = fade(yf)
    # noise components
    n00 = gradient(p[p[xi] + yi], xf, yf)
    n01 = gradient(p[p[xi] + yi + 1], xf, yf - 1)
    n11 = gradient(p[p[xi + 1] + yi + 1], xf - 1, yf - 1)
    n10 = gradient(p[p[xi + 1] + yi], xf - 1, yf)
    # combine noises
    x1 = lerp(n00, n10, u)
    x2 = lerp(n01, n11, u)  # FIX1: I was using n10 instead of n01
    return lerp(x1, x2, v)  # FIX2: I also had to reverse x1 and x2 here


def lerp(a, b, x):
    "linear interpolation"
    return a + x * (b - a)


def fade(t):
    "6t^5 - 15t^4 + 10t^3"
    return 6 * t**5 - 15 * t**4 + 10 * t**3


def gradient(h, x, y):
    "grad converts h to the right gradient vector and return the dot product with (x,y)"
    vectors = np.array([[0, 1], [0, -1], [1, 0], [-1, 0]])
    g = vectors[h % 4]
    return g[:, :, 0] * x + g[:, :, 1] * y


lin = np.linspace(0, 5, 512, endpoint=False)
x, y = np.meshgrid(lin, lin)  # FIX3: I had to swap x and y here

plt.imshow(perlin(x, y, seed=2), origin="upper")
plt.show()

perlin bing

“My” solution

I found the solution literally after 15 seconds of searching in this blog post which is both more elegant, concise and scalable.

view the code
def generate_perlin_noise_2d(shape, res):
    def f(t):
        return 6 * t**5 - 15 * t**4 + 10 * t**3

    delta = (res[0] / shape[0], res[1] / shape[1])
    d = (shape[0] // res[0], shape[1] // res[1])
    grid = np.mgrid[0 : res[0] : delta[0], 0 : res[1] : delta[1]].transpose(1, 2, 0) % 1
    # Gradients
    angles = 2 * np.pi * np.random.rand(res[0] + 1, res[1] + 1)
    gradients = np.dstack((np.cos(angles), np.sin(angles)))
    g00 = gradients[0:-1, 0:-1].repeat(d[0], 0).repeat(d[1], 1)
    g10 = gradients[1:, 0:-1].repeat(d[0], 0).repeat(d[1], 1)
    g01 = gradients[0:-1, 1:].repeat(d[0], 0).repeat(d[1], 1)
    g11 = gradients[1:, 1:].repeat(d[0], 0).repeat(d[1], 1)
    # Ramps
    n00 = np.sum(grid * g00, 2)
    n10 = np.sum(np.dstack((grid[:, :, 0] - 1, grid[:, :, 1])) * g10, 2)
    n01 = np.sum(np.dstack((grid[:, :, 0], grid[:, :, 1] - 1)) * g01, 2)
    n11 = np.sum(np.dstack((grid[:, :, 0] - 1, grid[:, :, 1] - 1)) * g11, 2)
    # Interpolation
    t = f(grid)
    n0 = n00 * (1 - t[:, :, 0]) + t[:, :, 0] * n10
    n1 = n01 * (1 - t[:, :, 0]) + t[:, :, 0] * n11
    return np.sqrt(2) * ((1 - t[:, :, 1]) * n0 + t[:, :, 1] * n1)


noise = generate_perlin_noise_2d((512, 512), (16, 16))

plt.imshow(noise, cmap="viridis")

perlin me


3. Diffusion PDE solver

Write a C++ class that implements a simple diffusion equation solver in 1D. The class should have the following methods:

  • constructor: takes the number of grid points, the grid spacing, and the time step
  • set_initial_condition: takes a function that sets the initial condition
  • set_boundary_conditions: takes a function that sets the boundary conditions
  • advance: advances the solution by one time step
  • get_solution: returns the current solution

Separate the class in its own header/cpp file. Also provide a CMakeLists.txt.

main.cpp
#include "diffusion_solver.h"
#include <algorithm>
#include <cmath>
#include <iostream>

void plot(const std::vector<double> &y) {
  std::cout << "y_max = " << *std::max_element(y.begin(), y.end()) << std::endl;
  const auto y_max = 1.0;
  const auto ny = 20;
  const auto nx = 50;
  std::vector<std::string> plot_str;
  for (std::size_t i{0}; i < ny; ++i) {
    plot_str.push_back(std::string(nx, ' '));
  }
  for (std::size_t i{0}; i < y.size(); ++i) {
    const auto x_i = static_cast<int>(nx * i / y.size());
    const auto y_i = static_cast<int>(ny * y[i] / y_max);
    if (x_i < nx && y_i < ny && x_i >= 0 && y_i >= 0) {
      plot_str[ny - y_i - 1][x_i] = '.';
    }
  }
  for (std::size_t i{0}; i < ny; ++i) {
    std::cout << plot_str[i] << std::endl;
  }
}

int main(int argc, char **argv) {
  // Define the problem parameters
  const int num_points = 100;
  const double domain_length = 1.0;
  const double delta_x = domain_length / (num_points - 1);
  const double delta_t = 0.00005;

  std::cout << "CFL: " << 2.0 * delta_t / (delta_x * delta_x) << std::endl;

  // Create the diffusion solver
  DiffusionSolver solver(num_points, delta_x, delta_t);

  // Set the initial condition
  solver.set_initial_condition(
      [](double x) { return std::pow(x * (1 - x) / 0.25, 5); });

  // Set the boundary conditions
  solver.set_boundary_conditions([](double x) { return 0; },
                                 [](double x) { return 0; });

  // Advance the solution in time
  const int num_time_steps = atoi(argv[1]);
  for (int i = 0; i < num_time_steps; i++) {
    solver.advance();
  }

  // Print the final solution
  const auto &solution = solver.get_solution();
  plot(solution);

  return 0;
}

Results

In both cases I tweaked the main.cpp file to add a plotter and interesting initial/boundary conditions, so that part will be the same for both solutions. Both AI models did a good job in providing the simplest possible boilerplate code. They both used std::function to pass initial/boundary conditions. They had slightly different approaches for actually enforcing the boundary conditions: ChatGPT passed two separate functions called for left/right boundaries, while Bing used a single function called on 0 and (npoints - 1) * dx.

ChatGPT forgot the pragma-clauses in the header, which is a minor thing in this particular problem, but when buildling a large project this might come back and bite you, so overall – not the best practice (-0.5 star). Two more caveats which took away a full star from their final rating is the fact they used the simplest Euler time-integrator (which, again, is fine for this problem, but not really viable for anything more complex), and more importantly, the fact they only allowed boundary conditions on the function itself, and not its derivative (i.e., no reflection boundaries). Overall, however, both models produced a very readable code, both of which could work great as a starting point.

Chat GPT-3

diffusion_solver.h
#include <functional>
#include <vector>

class DiffusionSolver {
public:
  DiffusionSolver(int num_points, double delta_x, double delta_t);
  void set_initial_condition(std::function<double(double)> func);
  void set_boundary_conditions(std::function<double(double)> left,
                               std::function<double(double)> right);
  void advance();
  const std::vector<double> &get_solution() const;

private:
  std::vector<double> m_solution;
  std::vector<double> m_previous;
  double m_delta_x;
  double m_delta_t;
  double m_coefficient;
  std::function<double(double)> m_left_bc;
  std::function<double(double)> m_right_bc;
};
diffusion_solver.cpp
#include "diffusion_solver.h"

DiffusionSolver::DiffusionSolver(int num_points, double delta_x, double delta_t)
    : m_solution(num_points), m_previous(num_points), m_delta_x(delta_x),
      m_delta_t(delta_t) {
  m_coefficient = delta_t / (delta_x * delta_x);
}

void DiffusionSolver::set_initial_condition(
    std::function<double(double)> func) {
  for (int i = 0; i < m_solution.size(); i++) {
    m_solution[i] = func(i * m_delta_x);
    m_previous[i] = m_solution[i];
  }
}

void DiffusionSolver::set_boundary_conditions(
    std::function<double(double)> left, std::function<double(double)> right) {
  m_left_bc = left;
  m_right_bc = right;
}

void DiffusionSolver::advance() {
  // Apply boundary conditions
  m_solution[0] = m_left_bc(0);
  m_solution[m_solution.size() - 1] =
      m_right_bc((m_solution.size() - 1) * m_delta_x);

  // Compute the next time step
  for (int i = 1; i < m_solution.size() - 1; i++) {
    m_solution[i] =
        m_previous[i] + m_coefficient * (m_previous[i - 1] - 2 * m_previous[i] +
                                         m_previous[i + 1]);
  }

  // Update the previous time step
  std::swap(m_solution, m_previous);
}

const std::vector<double> &DiffusionSolver::get_solution() const {
  return m_solution;
}
CMakeLists.txt
cmake_minimum_required(VERSION 3.10)
project(diffusion_solver)

set(CMAKE_CXX_STANDARD 11)

add_executable(diffusion_solver main.cpp diffusion_solver.cpp)
> cmake -B build && cd build && make -j && ./diffusion_solver 100

CFL: 0.9801
y_max = 0.840428



                      ......
                    ..      ..
                   ..        ..
                  ..          ..
                 ..            ..
                 .              .
                .                .
               .                  .
              .                    .
             ..                    ..
            ..                      ..
           ..                        ..
          ..                          ..
         ..                            ..
       ..                                ..
     ...                                  ...
.....                                        .....

Bing Chat

diffusion_solver.h
#ifndef DIFFUSIONSOLVER_H
#define DIFFUSIONSOLVER_H

#include <functional>
#include <vector>

class DiffusionSolver {
public:
  DiffusionSolver(int n_points, double dx, double dt);
  void set_initial_condition(std::function<double(double)> f);
  void set_boundary_conditions(std::function<double(double)> f);
  void advance();
  std::vector<double> get_solution() const;

private:
  int n_points;
  double dx;
  double dt;
  std::vector<double> u;
};

#endif
diffusion_solver.cpp
#include "diffusion_solver.h"

DiffusionSolver::DiffusionSolver(int n_points, double dx, double dt)
    : n_points(n_points), dx(dx), dt(dt), u(n_points) {}

void DiffusionSolver::set_initial_condition(std::function<double(double)> f) {
  for (int i = 0; i < n_points; ++i) {
    u[i] = f(i * dx);
  }
}

void DiffusionSolver::set_boundary_conditions(std::function<double(double)> f) {
  u[0] = f(0);
  u[n_points - 1] = f((n_points - 1) * dx);
}

void DiffusionSolver::advance() {
  std::vector<double> u_new(n_points);
  for (int i = 1; i < n_points - 1; ++i) {
    u_new[i] = u[i] + dt / (dx * dx) * (u[i + 1] - 2 * u[i] + u[i - 1]);
  }
  u = u_new;
}

std::vector<double> DiffusionSolver::get_solution() const { return u; }
CMakeLists.txt
cmake_minimum_required(VERSION 3.10)
project(diffusion_solver)

set(CMAKE_CXX_STANDARD 11)

add_executable(diffusion_solver main.cpp diffusion_solver.cpp)
> cmake -B build && cd build && make -j && ./diffusion_solver 100

CFL: 0.9801
y_max = 0.840428

                      ......
                    ..      ..
                   ..        ..
                  ..          ..
                 ..            ..
                 .              .
                .                .
               .                  .
              .                    .
             ..                    ..
            ..                      ..
           ..                        ..
          ..                          ..
         ..                            ..
       ..                                ..
     ...                                  ...
.....                                        .....

4. Extract white regions with ffmpeg

I have a video file called christmas.mp4. I want to use ffmpeg to extract white regions from it. Remove everything but the white regions by applying a transparent mask. The output video must be called christmas-white.mp4, and only the white regions from the original footage must be visible. The final video must contain an alpha channel.

I used the following footage for this task.

Results

Since I don’t really have any experience with ffmpeg, I often find myself looking for very specific set of commands and flags in forums and documentations, and I thought this would be a great test for both AI models. The results, unfortunately, didn’t really yield any useful results. So I’m giving both Chat GPT-3 and Bing Chat a fail.

Chat GPT-3

First attempt:

ffmpeg -i christmas.mp4 -filter_complex "colorkey=0xFFFFFF:0.1:0.2,format=yuva420p" -c:v libx264 -c:a copy -movflags +faststart -preset ultrafast -crf 18 christmas-white.mp4

Second attempt:

ffmpeg -i christmas.mp4 -filter_complex "[0:v]colorkey=0xFFFFFF:0.1:0.2[ckout];[ckout]format=yuva420p[ckout_alpha];[0:v][ckout_alpha]overlay[out]" -map "[out]" -map 0:a -c:v libx264 -c:a copy -movflags +faststart -preset ultrafast -crf 18 -pix_fmt yuv420p christmas-white.mp4

Bing Chat

First attempt:

ffmpeg -i christmas.mp4 -vf "chromakey=0xffffff:0.1:0.5" -c:a copy christmas-white.mp4

Second attempt:

ffmpeg -i christmas.mp4 -filter_complex "[0:v]split[v1][v2];[v1]lutyuv='u=128:v=128:y=maxval'[mask];[v2][mask]overlay" -c:a copy christmas-white.mp4

Third attempt:

ffmpeg -i christmas.mp4 -filter_complex "[0:v]split[v1][v2];[v1]geq='if(gt(lum(X,Y),235),255,0)',format=gray[mask];[v2][mask]alphamerge" -c:a copy christmas-white.mp4

It got desperate:

You can try posting your question on forums or websites dedicated to ffmpeg or video editing, such as the ffmpeg-user mailing list or the ffmpeg subreddit. These communities may be able to provide more detailed and specific guidance on how to achieve your desired result.

My solution

I have no experience with ffmpeg whatsoever, so I had to spend 5-10 mins searching through the documentation and forums. This is not ideal, since it doesn’t really make an alpha channel and instead just greys out the remaining regions and turns the white regions into black, but it kinda works, and also provides a way to tweak the sensitivity of the color keying.

ffmpeg -i christmas.mp4 -filter_complex "[0:v]colorkey=0xFFFFFF:0.1:0.6,split[m][a];[m]drawbox=t=fill[mm];[a]geq=lum_expr='(p(X,Y)*255)':a=0[aa];[aa]format=yuva420p, alphaextract[aaaa];[aaaa][mm]overlay=format=yuv420[out]" -map "[out]" -c:v libx264 -preset medium -crf 18 -c:a copy -movflags +faststart -y christmas-black.mp4

Final thoughts

After this set of tests I was actually more surprised than disappointed. As I mentioned, the goal was to produce a reasonable boilerplate code that can later be improved upon on timescales comparable to or better than the time it would take to write the code myself or too search it somewhere on the web. For people who are just learning to code this is probably good-enough in most scenarios to get things started. The last ffmpeg test was a bit of a letdown, since I expected that less tedious tasks would be easier to solve with AI, but I guess that’s not the case (also let’s just agree that the ffmpeg API is borderline chaotic).

I find the GitHub Copilot way more useful in daily tasks, as it can really parse the context well-enough to provide with reasonable completions that save me tons of coding time. That being said, of course, after it introduced some very subtle bugs in my code which I spent hours to find and debug, I now find myself being a lot more careful when using it (which is probably a good thing), and triple checking all the lines before actually moving on. At this point, I think the level of GPT-3 programming abilities are comparable to that of a starting early-undergrad programmer, which, honestly, is not bad at all, although I wouldn’t trust it with launching rockets or running its code on millions of supercomputer nodes. Perhaps with the advent of GPT-4 we’ll see some improvements in this area.

cd ~