I'm working on a TL;DR bot for Lemmy, powered by GPT-3.5

azayrahmad, 9 months ago

This is a great idea! What if there’s a post about, say, a movie review, then it includes a link to the movie’s imdb or letterboxd. Would it summarized the link instead of the review?

sisyphean, 9 months ago

It would summarize the link. Unfortunately that’s an edge case where the bot doesn’t do what you mean.

StudioLE, 10 months ago

What language is it written in, and have you considered sharing the source?

danc4498, 10 months ago

I'm always curious about using GPT like that. Does it cost money to send requests like this into GPT?

sisyphean, 10 months ago

It does unfortunately, see here:

https://openai.com/pricing

I limited it to 100 summaries / day, which adds up to about $20 (USD) per month if the input is 3000 tokens long and the answer is 1000.

Using it for personal things (I buildt a personal assistant chatbot for myself) is very cheap. But if you use it in anything public, it can get expensive quickly.

Hexarei, 10 months ago

Have you considered using a self-hosted instance of GPT4All? It's not as powerful, but for something like summarizing an article it could be plenty - And importantly, much, much cheaper.

sisyphean, 10 months ago

I haven’t yet looked into it, but the screencast on its website looks really promising! I have a lot on my plate right now so I think I’ll release it first with the GPT-3.5 integration, but I’ll definitely try GPT4All later!

webghost0101, 10 months ago (edited 10 months ago)

How does GPT4ALL work exactly? Surely its no where near the quality of real GPT4.

If i check on https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard The 13B model is below some of the llama models. At this point Falcom 40B is what you want to have the best quality llm running locally

Hexarei, 10 months ago

It's just an open source LLM that you can run on your own hardware; I haven't looked into it a ton tbh - but if I saw $20/month for 100 requests/day I'd be immediately looking for a way to run it on my own hardware

webghost0101, 10 months ago

Its kinda debatable, while yes I wouldnt wanna pay that either, ive been following the local working llms, Gpt4all stroke me as not bad but not all that special or amazing (compared to 2021 there all magic though) the naming seems a very little bit misleading with gpt4 as the world most advanced known model. All the models on tbe huggingface page i send work can work locally but at best there gpt-3 level competitors.

rr7, 10 months ago

I love it! 👑👑

sisyphean, 10 months ago

Thank you:)

danc4498, 10 months ago

@TLDRBot

sisyphean, 10 months ago

It doesn't work yet, the screenshots are from a test Lemmy instance

danc4498, 10 months ago

Good bot

sisyphean, 10 months ago

Aww thank you, it warms my circuitry ☺️