I spent all day comparing Bard vs ChatGPT 4. Here are the results (Software Developer Questions)

I’m a software engineer at a startup with impossible deadlines - I’ve used GPT4 for months to generate huge amounts app/server code, and much like your IDE, once you learn to use these tools you don’t want to go back to the days without it.

Speed

  • Bard is very fast- similar to GPT 3.5 Turbo
  • You need to multitask two GPT4 instances side by side to compensate for how slow GPT4 can be

Reliability

  • Bard lies and makes up fake API calls more than GPT4

UI

  • Bard UI is garbage - You have to keep manually scrolling down the chat window, and for some reason the largest button on the page is “stop” (???)
  • You can tell Bard to modify its response to be longer/shorter and a few other options - I thought this would be useful, but it never ended up helping

Memory

  • Bard has really short memory - Forgets details from last response!
  • GPT4 memory is also unreliable, any details that are important you have to repeat

Intelligence

  • GPT4 is objectively smarter

Internet Search

  • GPT4 Internet search is garbage
  • Bard has “Verify with Google” - I had high hopes for this, but never actually had a use for it

Willingness to give full code

  • GPT4 is bad, but Bard is worse. Both need to be begged/threatened to return more than 100 lines of directly paste-able code.

Generating Useful Code

  • Bard can give more concise medium complexity functions

Adding tougher features

  • Bard hallucinates and lies

Dealing with lies

  • When you tell GPT something doesn’t work, GPT will try something else
  • When you tell Bard something doesn’t work, Bard will lie, claim to fix it, then give back the same code

Following Instructions

  • GPT4 sometimes doesn’t follow instructions, but improving the prompt will fix that. Bard will happily ignore instructions, as clear as they may be.

Summary:

  • GPT4 is still objectively better than Bard. Quite frankly, the prompts Bard couldn’t handle, GPT3.5 could.
  • The cons of GPT can be worked around, but for Bard, it’s almost faster to do it yourself. Unless Bard was used like Copilot for short 1-2 lines of autocomplete, I wouldn’t trust it.

PS: If you’re not using AI yet for development, I highly recommend it - It’s like using an IDE instead of Notepad. AI can easily 2-3x your output, but you have to learn how it works so you can prompt it correctly, and you have be good at fixing its mistakes.

BallShapedMan,
@BallShapedMan@lemmy.world avatar

This is fantastic! Thank you for putting in the work.

Back in things like Google maps and phone assistant were new there had to be YouTube competitions to see which was better at a given set of tasks. I say that say maybe this is your way to Internet fame?

Rentlar,

The largest button on the page is “stop” (???)

It’s there to be sure we find that button when Bard gets aspirations to take over the world or initiate thermonuclear warfare I suppose.

popcar2, (edited )

I still can’t understand why Google keeps hyping up Bard and then releasing it at a poor state just to ruin their reputation. First, we had:

  • Bard 1, which was hyped up to be the ChatGPT successor. It turned out to be really bad.
  • Bard 2.0, a massive update that was hyped up to make Bard so much better. It turned out to still be pretty bad (but in fairness it was a minor improvement).
  • Google Gemini, their massive response to GPT 4 that was, on paper, the best LLM in the world. They finally integrated it into Bard last month and… It’s still not great. I could not tell an immediate difference between this and the old Bard. Oh, and the videos they used to advertise Gemini Ultra were fake.

I’m not going to armchair analyze a hugely successful company, but from my point of view it really shows how mismanaged Google has been in the past decade. Failed projects upon cancelled projects upon increasingly frustrated employees.

/rant. Anyways, you should consider using Perplexity if you want something with search capabilities, I’ve had decent success there. Claude is also significantly better than Bard, but they made free usage very limited lately. Might be a good option if you’re willing to pay.

Deckweiss,

Have you tried the custom/community AI in ChatGPT4? I have found “AutoExpert(Dev)” to be significantly better at programming than default ChatGPT4.

You find them in the “Explore GPTs” and you might need to enable it in the settings before you see that.

ck_,

I’m a software engineer […]

No… No, you’re not …

You are that guy who takes 50 guesses at the easy trivial pursuit questions and then claims you knew the answer all along.

Carighan,
@Carighan@lemmy.world avatar

Since we just had a discussion about how aggressive the discussions on Lemmy have recently become, where did this come from? 😅 Am I missing some context here, or is this the most weirdly unprovoked attack?

Lmaydev,

All the cool kids hate LLMs essentially.

BaroqueInMind,
BaroqueInMind avatar

Maybe people should stop acting like peal clutching, limp-wrist, fragile little bitches when reading naughty words on the internet or when reading a critique about something/someone. Or maybe we should all talk polite and act kind to one another even when it's not truly genuine or necessary. Both are possible.

Lmaydev,

Hmmm be a giant asshole for no reason or be kind to others. Pretty hard to decide which is better hehe

Lmaydev,

These tools are game changers imo.

Personally I don’t use them to write code but to query how to do things.

It’s a massive productivity booster for me.

LemmyIsFantastic,

Blah blah blah blah blah blah blah blah blah blah

Nobody cares how good you are at algorithm efficiency. Most coders like you are terrible to work with and are kept on for niche needs.

OsrsNeedsF2P, (edited )

github.com/dginovker, gitlab.com/dginovker

@me bro. I’ve been writing open source software for 5 years before GPT came along. Keep up with the tooling. I recommend you try it before you knock it.

ck_,

I have 15+ years of experience being a software engineer and now I’m making a mid-six-figure income by going into companies who staff their “engineering department” with people who have five years or less of experience and cannot write a line of code without internet access to save their lives. So by all means go ahead and continue down the road you are going. We thought that stack overflow would guarantee a stable business but now that “AI” has come into play, we can’t even keep up with demand. We’ll probably raise prices by 25% next quarter. By the time that AI can actually produce decent results, I can probably retire twice over.

OsrsNeedsF2P,

Imagine making 500k TC but not contributing to open source

ck_,

When I started doing open source software, that meant posting tarballs to Usenet and mailing list, occasionally mailing someone a hard floppy on request. I don’t have a github profile sparkling with emojies, but I think I’m doing all right.

grilledcheesecowboy,

And now you're here screaming at clouds and bitching about why kids aren't as good at writing FORTRAN as you are. The wheels of time have been cruel to you.

Before you start internet flexing about "all the money you make" or "how great you are at coding", no one cares. Those things aren't real on the internet, all that's real is your shitty scared by indoor plumbing boomer comments.

Kata1yst,
Kata1yst avatar

What a weirdly unjustified, baseless attack on someone contributing a useful conversation. And when presented with evidence that you're not just being a huge jerk, that you're entirely wrong, you get defensive and continue to denegrate OP?

Shame on you. If you have had half the illustrious career you claim to have, you should have worked in enough places and with enough people to know when to eat crow.

Infighting and personal attacks like this from positions of false authority like yours are exactly why people have such low opinions of programmers as members of society.

pineapplelover,

I’ve also given bing gpt4 some coding related stuff and for first year comp sci, it can pretty much do it. Anything beyond, you will need very specific instructions, at that point, you can probably code it yourself. If you aren’t specific, it will give you something entirely wrong, claim it did your prompt, gives lots of errors many of the time. You have to baby walk it a lot of the time.

Additionally, it feels way more restrictive than a few months back. Seems like it gives you generic prompts and not even try to spin it up a little.

bungle_in_the_jungle,

You should try copilot if you haven’t yet.

OsrsNeedsF2P,

Copilot is great. I stopped paying for it after using ChatGPT so much (one subscription is expensive enough, as-is!) but I do miss it. Maybe I’ll buy it again…

Daxtron2,

I’m working on convincing my boss to get licenses for the team lol. I just use a personal license but it saves a lot of typing and usually has a good idea of what I want.

gears,

I managed to convince mine, and it’s been great. I miss it at home when I’m doing personal projects

OsrsNeedsF2P,

It’s honestly worth it tbh, even paying out of pocket. Our salaries are how much an hour? Save that and go home early 😄

I’m currently eyeing this PR to CodeGPT, a free plugin alternative to ChatGPT - This PR would add the autocomplete feature: github.com/carlrobertoh/CodeGPT/pull/333

CodeGPT lets you run a model locally, and my work computer has a GFX card strong enough to do that. The local models have gotten as good as GPT3.5, so needless to say the money-saving part of me is very excited about this!

Daxtron2,

Haha now I have to convince him to upgrade our laptops to gaming laptops too

Adanisi,
@Adanisi@lemmy.zip avatar

Copilot regurgitates code verbatim and strips the licensing (see that one video of it spitting out Quake inverse sqrt). Don’t use it if you care about legality.

OsrsNeedsF2P,

Quake inverse sqrt is one of the most famous algos in the world. Although I agree, if politicians actually stood for public interest they would force any AI derivative work to be SSPL or AGPL

Adanisi,
@Adanisi@lemmy.zip avatar

Yes, its very famous. It’s also GPLed and Copilot had no right to use it in that way, and strip the license.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • programming@programming.dev
  • DreamBathrooms
  • mdbf
  • ngwrru68w68
  • magazineikmin
  • thenastyranch
  • rosin
  • khanakhh
  • osvaldo12
  • Youngstown
  • slotface
  • Durango
  • kavyap
  • InstantRegret
  • tacticalgear
  • anitta
  • ethstaker
  • provamag3
  • cisconetworking
  • tester
  • GTA5RPClips
  • cubers
  • everett
  • modclub
  • megavids
  • normalnudes
  • Leos
  • JUstTest
  • lostlight
  • All magazines
    •