While LLMs have been used for… a lot, it seems like this use might be one where it’s not only reliable but it appears to outperform existing methods of image compression. Being able to cram more data into less space tends to lead to interesting developments, so I will be keeping my eye on this....
That's not what lossless data compression schemes do:
In lossless compression the general idea is to create a codebook of commonly occuring patterns and use those as shorthand.
For example, one of the simplest and now ancient algorithms LZW does the following:
Initialize the dictionary to contain all strings of length one.
Initialize the dictionary to contain all strings of length one.
Emit the dictionary index for W to output and remove W from the input.
Add W followed by the next symbol in the input to the dictionary.
repeat
Basically, instead of rewriting long sequences, it just writes down the index into an existing dictionary of already seen sequences.
However, once this is done, you now need to find an encoding that takes your characterset (the original characters+the new dictionary references) and turns it into bits.
It turns out that we can do this optimally: Using an algorithm called Arithmetic coding we can align the length of a bitstring to the amount of information it contains.
"Information" here meaning the statistical concept of information, which depends on the inverse likelihood a certain character is observed.
Logically this makes sense:
Let's say you have a system that measures earthquakes. As one would expect, most of the time, let's say 99% of the time, you will see "no earthquake", while in 1% of the cases you will observe "earthquake".
Since "no earthquake" is a lot more common, the information gain is relatively small (if I told you "the system said no earthquake", you could have guessed that with 99% confidence: not very surprising).
However if I tell you "there is an earthquake" this is much more important and therefore is worth more information.
From information theory (a branch of mathematics), we know that if we want to maximize the efficiency of our codec, we have to match the length of every character to its information content. Arithmetic coding now gives us a general way of doing this.
However, we can do even better:
Instead of just considering individual characters, we can also add in character pairs!
Of course, it doesn't make sense to add in every possible character pair, but for some of them it makes a ton of sense:
For example, if we want to compress english text, we could give a separate codebook entry to the entire sequence "the" and save a ton of bits!
To do this for pairs of characters in the english alphabet, we have to consider 26*26=676 combinations.
We can still do that: just scan the text 600 times.
With 3 character combinations it becomes a lot harder 26*26*26=17576 combinations.
But with 4 characters its impossible: you already have half a million combinations!
In reality, this is even worse, since you have way more than 26 characters: you have things like ", . ? ! and your codebook ids which blow up the size even more!
So, how are we supposed to figure out which character pairs to combine and how many bits we should give them?
We can try to predict it!
This technique, called [PPM](Prediction by partial matching) is already very old (~1980s), but still used in many compression algorithms.
The important trick is now that with deep learning, we can train even more efficient estimators, without loosing the lossless property:
Remember, we only predict what things we want to combine, and how many bits we want to assign to them!
The worst-case scenario is that your compression gets worse because the model predicts nonsensical character-combinations to store, but that never changes the actual information you store, just how close you can get to the optimal compression.
The state-of-the-art in text compression already uses this for a long time (see Hutter Prize) it's just now getting to a stage where systems become fast and accurate enough to also make the compression useful for other domains/general purpose compression.
The car is the same as last week.
You have to remember that this is a track that verstappen really doesn't like: last year's race at Singapore was also his worst.
Usually verstappen drives ~3 tenths faster than Perez, which, if he did that this week, would also put him up there....
IMO this is less of a case that the car is worse and more that verstappen isn't able to get 100% from his car.
24, always driven manual, EU.
From my experience most people in the EU can or at least could: This is because many (if not all, not sure) countries make a distinction between manual and automatic licenses (see e.g. https://www.learn-automatic.com/qualified/automatic-driving-licence/).
I.e. if you want to drive manual, you have to take the test manual, but if you take the test on manual transmission, you are allowed to drive automatics as well.
No, it's built into the protocol: think of it like as if every http request forces you to attach some tiny additional box containing the solution to a math puzzle.
The twist is that you want the math puzzle to be easy to create and verify, but hard to compute. The harder the puzzle you solve, the more you get prioritized by the service that sent you the puzzle.
If your puzzle is cheaper to create than hosting your service is, then it's much harder to ddos you since attackers get stuck at the puzzle, rather than getting to your expensive service
They are: their car is just a dog in the actual race. From a pure qualifying pace POV they are a lot better, with hulkenberg being able to get the car into q3 quite consistently.
That's also what makes them seem better than they really are: hulk qualifying in p8 (great) and then tumbles down to p16 by the end of the race (usually because they have to stop more often or at least to worse tires since their tire deg is abismal)
Standard lossless compression (without further assumptions) is already very close to being as optimal as it can get: At some point the pure entropy of these huge datasets just is not containable anymore.
The most likely savior in this case would be procedural rendering (i.e. instead of storing textures and meshes, you store a function that deterministically generates the meshes and textures). These already are starting to become popular due to better engine support, but pose a huge challenge from a design POV (the nice e.g. blender-esque interfaces don't really translate well to this kind of process).
No he doesn't?
Don't get me wrong there are many places where the paper can be wrong (eg fig 1 or their magnetism exceptionally looking more similar to diamagnetism than superconductivity) but you are mixing him up with Ranga Dias who has had a history of data fabrication.
Dias has nothing to do with this paper though.
Allegations of data fabrication have sparked the retraction of multiple papers from Ranga Dias, a researcher who claimed discovery of a room-temperature superconductor
Not really: you have to keep in mind the amount of expertise and ressources that already went into silicon, as well as the geopolitics and sheer availability of silicon. The closest currently available competitor is probably gallium arsenide. That has a couple of disadvantages compared to silicon
It's more expensive (both due to economies of scale and the fact that silicon is just much more abundant in general)
GaAs crystals are less stable, leading to smaller boules.
GaAs is a worse thermal conductor
GaAs has no native "oxide" (compare to SiO₂) which can be directly used as an insulator
GaAs mobilities are worse (Si is 500 vs GaAs 400), which means P channel FETs are naturally slower in GaAs, which makes CMOS structures impossible
GaAs is not a pure element, which means you get into trouble with mixing the elements
You usually see GaAs combined with germanium substrates for solar panels, but rarely independently of that (GaAs is simply bad for logic circuits).
In short: It's not really useful for logic gates.
Germanium itself is another potential candidate, especially since it can be alloyed with silicon which makes it interesting from an integration point-of-view.
SiGe is very interesting from a logic POV considering its high forward and low reverse gain, which makes it interesting for low-current high-frequency applications. Since you naturally have heterojunctions which allow you to tune the band-gap (on the other hand you get the same problem as in GaAs: it's not a pure element so you need to tune the band-gap).
One problem specifically for mosfets is the fact that you don't get stable silicon-germanium oxides, which means you can't use the established silicon-on-insulator techniques.
Cost is also a limiting factor: before even starting to grow crystals you have the pure material cost, which is roughly $10/kg for silicon, and $800/ kg for germanium.
That's why, despite the fact that the early semiconductors all relied on germanium, germanium based systems never really became practical: It's harder to do mass production, and even if you can start mass production it will be very expensive (that's why if you do see germanium based tech, it's usually in low-production runs for high cost specialised components)
There's some research going on in commercialising these techniques but that's still years away.
I hope that heads roll at haas for that disaster. There's one thing making wrong choices in high-stakes scenarios, but they sent out their drivers too late twice within 24h. That's just an unforced, unexplainable blunder. If I were gene haas I'd be furious: spend 100 MILLION dollars to develop a car and they don't even manage to get it around the track once.
This is doubly bad considering that sprints are one of their most reliable places to get points considering that their tire wear doesn't affect them too much over shorter distances.
They might as well pack up and go home now to conserve their parts since at this point there not going to achieve anything anyways.
They will make it open source, just tremendously complicated and expensive to comply with.
In general, if you see a group proposing regulations, it's usually to cement their own positions: e.g. openai is a frontrunner in ML for the masses, but doesn't really have a technical edge against anyone else, therefore they run to congress to "please regulate us".
Regulatory compliance is always expensive and difficult, which means it favors people that already have money and systems running right now.
There are so many ways this can be broken in intentional or unintentional ways. It's also a great way to detect possible e.g. government critics to shut them down (e.g. if you are Chinese and everything is uniquely tagged to you: would you write about Tiananmen square?), or to get monopolies on (dis)information.
This is not literally trying to force everyone to get a license for producing creative or factual work but it's very close since you can easily discriminate against any creative or factual sources you find unwanted.
In short, even if this is an absolutely flawless, perfect implementation of what they want to do, it will have catastrophic consequences.
“Our primary conclusion across all scenarios is that without enough fresh real data in each generation of an autophagous loop, future generative models are doomed to have their quality (precision) or diversity (recall) progressively decrease,” they added. “We term this condition Model Autophagy Disorder (MAD).”...
That paper makes a bunch of(implicit) assumptions that make it pretty unrealistic: basically they assume that once we have decently working models already, we would still continue to do normal "brain-off" web scraping.
In practice you can use even relatively simple models to start filtering and creating more training data:
Think about it like the original LLM being a huge trashcan in which you try to compress Terrabytes of mostly garbage web data.
Then, you use fine-tuning (like the instruction tuning used the assistant models) to increases the likelihood of deriving non-trash from the model (or to accurately classify trash vs non-trash).
In general this will produce a datasets that is of significantly higher quality simply because you got rid of all the low-quality stuff.
This is not even a theoretical construction: Phi-1 (https://arxiv.org/abs/2306.11644) does exactly that to train a state-of-the-art language model on a tiny amount of high quality data (the model is also tiny: only half a percent the size of gpt-3).
Previously tiny stories https://arxiv.org/abs/2305.07759 showed something similar: you can build high quality models with very little data, if you have good data (in the case of tiny stories they generate simply stories to train small language models).
In general LLM people seem to re-discover that good data is actually good and you don't really need these "shotgun approach" web scrape datasets.
Daniel Ricciardo will make a sensational return to front-line Formula 1 racing at the upcoming Hungarian Grand Prix, with the Australian set to replace Nyck de Vries at the AlphaTauri team for the rest of the season.
I think you also have to keep in mind the position that de Vries and redbull is in:
Redbull is looking for a second verstappen-level driver. That's always been the case not only for redbull, but all tier 1 teams: Their aspirations are championships, not points or even podiums.
De Vries is a 28 year old rookie. That's usually the time that drivers retire or lean on their superior experience to make up for their loss in reaction speed and overall pace. The problem is that De Vries has no experience, while being older than Verstappen by close to three years. The fact that he got to race at all is a miracle: He would have to beat Tsunoda every week by quite a margin to become relevant for RedBull. If he doesn't become relevant for redbull, then why have him at alpha tauri?
Meanwhile they have a young driver in the form of tsunoda which exists in a limbo due to him having nothing to compare against: He could be the fastest driver on the planet in a trash car, or he could be underdelivering without anyone noticing due to the lack of comparison.
This is bad for two reasons:
you don't know whether tsunoda is an option for redbull
you have no idea how good alpha tauri is over all, which is doubly bad considering that they want to make major changes to how alpha tauri operates.
On the other hand, you have a perfectly good Ricciardo sitting on his hands that performed really well at silverstone. Realistically, you aren't going to lose anything from having Riccardo drive the rest of the season compared to having de Vries drive, but you have to potential upside of more context to the quality of tsunoda and the team, which you wouldn't get otherwise.
In general I'm more suprised that they ever gave De Vries a chance considering his age and the context to his big achievements:
In formula 2 his stiffest competitor was Nicholas Latifi (He won with 266 vs Latifi's 214 points) in what can be described as a dud year after the majority of now F1 mainstays had already graduated (he also needed 3 years to win F2, which is never a good sign).
If you have ever seen an formula E race, you will notice that it is quite a chaotic crash-fest with very weird rules and other nonsense. Just not crashing and not driving to quickly can get you really far by surviving the carbon-fiber mayhems and fuel-conservation issues.
To put it into perspective, here are the race records in the year that De Vries won formula E [1st, 9th, retired, retired, 1st, 16th, retired, 9th, retired, 13th, 18th, 2nd, 2nd, 22nd, 8th] or, in short if we ignore all DNFs we get a mean position of 9th!
In short, there's a reason why Mercedes never even tried to get him an F1 spot: He's not a bad driver, but being "not a bad driver" is insufficient for a top team like mercedes and redbull. There's little incentive to put him into any car, even less so nowadays considering his age.
Everything using the activityPub standard has open likes (see https://www.w3.org/TR/2018/REC-activitypub-20180123/ for the standard), and logically it makes sense to do this to allow for verification of "likes":
If you did not do that, a malicious instance could much more easily just shove a bunch of likes onto another instance's post, while, if you have "like authors" it's much easier to do like moderation.
Effectively ActivityPub treats all interactions like comments, where you have a "from" and "to" field just like email does (just imagine you could send messages without having an originator: email would have unusable levels of spam and harassment).
Specfically, here is an example of a simple activity:
As you can see this has a very "email like" structure with a sender, receiver, and content. The difference is mostly that you can also publish a "type" that allows for more complex interactions (e.g. if type is comment, then lemmy knows to put it into the comments, if type is like it knows to put it to the likes, etc...).
The actual protocol is a little more complex, but if you replace "ActivityPub" with "typed email" you are correct 99% of the time.
The different services, like lemmy, kbin, mastodon, or peertube are now just specific instantiations of this standard. E.g. a "like" might have slightly different effects on different services (hence also the confusion with "boosting" vs "liking" on kbin)
Its acquirer (Bending Spoons) has taken over operations. They’ve also hiked subscriptions prices and told customers they intend to use new revenues to pay for new features. How they intend to do that without any staff is something I would like to know about....
It really depends on what you want: I really like obsidian which is cross-platform and uses basically vanilla markdown which makes it easy to switch should this project go down in flames (there are also plugins that add additional syntax which may not be portable, but that's as expected).
There's also logseq which has much more bespoke syntax (major extensions to markdown), but is also OSS meaning there's no real danger of it suddenly vanishing from one day to the next.
Specifically Logseq is much heavier than obsidian both in the app itself and the features it adds to markdown, while obsidian is much more "markdown++" with a significant part of the "++" coming from plugins.
In my experience logseq is really nice for short-term note taking (e.g. lists, reminders, etc) and obsidian is much nicer for long-term notes.
Some people also like notion, but i never got into that: it requires much more structure ahead of time and is very locked down (it also obviously isn't self-hosted). I can see notion being really nice for people that want less general note-taking and more custom "forms" to fill out (e.g. traveling checklists, production planning, etc..).
Personally, I would always go with obsidian, just for the piece of mind that the markdown plays well with other markdown editors which is important for me if I want a long-running knowledge base.
Unfortunately I cannot tell you anything with regards to collaboration since I do not use that feature in any note-taking system
Should have been done a long time ago. Even adding and removing gravel traps where they currently have the blue concrete should be within the realms of possibility for an F1 gp if they want both F1 and MotoGP (consider that places like Baku literally pave their historical cobblestone and then un-pave it after the gp)
That depends on the size of the insurance: keep in mind that, for the most part, kbin is just a list of txt files. 2gb of ram sounds like a lot less than it is since people are used to desktops that have all sorts of additional stuff running on the side which pushes up the overall system consumption
They choose to do this. Delicious has historically been a point and click developer, but they wanted to diversify, especially since their previous title "pillars of the earth" flopped. They first tried their have at rts with "a year of rain" which is simply not that good, and then looked into Gollum.
You also can't raid make the argument that the project was rushed out the door, considering the game was supposed to release in 2021 (two years ago).
They tried something they had no experience in, not through coercion but because they wanted to, and produced a game of shockingly low quality. Since this wasn't the first flop, but just the latest in a huge series of flops, (though it was the most expensive and high profile one) the studio closed.
I know this is a silly question, especially as I'm currently focused on local-only feeds, but there's just one part of the Fediverse I wish would get more love: PeerTube. While there aren't a lot of people actively looking for alternative video hosts outside the big guys (Twitch, YouTube, Dailymotion, etc.), it would be nice to...
Peertube is inherently very scalable with relatively little cost due to an artifact of all social media platforms: Most of the traffic is driven by a tiny amount of videos/magazines/etc...
For services like youtube, you can use this as a way to quickly cache data close to the place it's going to be streamed: e.g. Netflix works with ISPs to install small servers at their locations to lessen the burden on their (and the ISPs) systems.
But with centralised systems you can only push this so far since ultimately everything is still concentrated at one central location.
Hypothetically, if you could stop this super-linear scaling for each user (you need to pay per user plus overhead generated from managing them at scale), you could easily compete against the likes of youtube simply because, at sufficient scale, all the other effects get ammortized away.
Peertube does exactly this by serving the videos as webtorrents: essentially this means that for every "chunk" of a video you downloaded, you also host that chunk for other people to download. That means that peertube itself theoretically only has to host every unique video once (or less than once since the chunks are in the network for a while), meaning you rid yourself of the curse of linear user scaling against users and only scale sub-linearly with the number of unique videos (how sub-linear depends on the lifetime for your individual torrents; i.e. how long a single video chunk stays available for others).
The costs that remain for every peertube instance is essentially the file hosting costs (and encoding the video, but that also only scales in the number of videos and could be pushed onto the uploader using WASM video encoders).
Storage itself isn't cheap, but also not ungodly expensive (especially since you can ammortize the costs over a long time as you platform grows with storage prices in a continual massive decline).
Platforms like Netflix and youtube cannot do this because
Netflix is a paid-service and people don't want to do the hosting job for netflix after having already paid for the service
Youtube has to serve adds which is incompatible with the "users host the content" method
In general torrenting is a highly reliable and well tested method that scales fantastically well to large data needs (it quite literally becomes more efficient the more people use it)
And don't forget that even after that you still have to watch baked-in "This video is sponsored by <insert shady company here>" adds since the actual revenue that gets passed to creators from youtube is so low that to keep the ship afloat they have to look for additional revenue streams.
I discovered yesterday evening that Lemmy.ml is blocking all inbound ActivityPub requests from /kbin instances. Specifically, a 403 'access denied' is returned when the user agent contains "kbinBot" anywhere in the string. This has been causing a cascade of failures with federation for many server owners, flooding the message...
Go to the relevant domain's front page (e.g https://kbin.social/d/kbin.social for kbin.social).
The URL scheme is "https://kbin.social/d/DOMAINHERE" assuming you are currently on kbin.social.
On the right in the sidebar you can see "Domain" and below that options to subscribe or to block.
Really it's the same thing as magazines, just that you generally don't visit the domain itself.
While the inability to source is a huge problem, but you also have to keep in mind that complaining about AI has other objective beyond the obvious "AI bad".
it's marketing: "Our thing is so powerful it could irreparably change someone's life" is still advertising even if that irreparable change is bad. Saying "AI so powerful it's dangerous" just sounds less advertis-y than "AI so powerful you cannot not invest in it" despite both leading to similar conclusions (you can look back at the "fearvertising" done during the original AI boom: same paint, different color)
it's begging for regulatory zeals to be put into place: Everyone with a couple of millions can build an LLM from scratch. That might sound like a lot, but it's only getting cheaper and it doesn't need highly intricate systems to replicate. Specifically the ability to finetune a large model with few datapoints allows even open-source non-profits like OpenAssistant to compete against the likes of google and openai: Google has made that very explicit in their leaked We have no moat memo. This is why you see people like Sam Altman talking to congress about the dangers of AI: He has no serious competetive advantage and hopes that with sufficient fear-mongering he can get the government to give him one.
Complaining about AI is as much about the AI as it is about the economical incentives behind AI.
AI language models can exceed PNG and FLAC in lossless compression, says study (arstechnica.com)
While LLMs have been used for… a lot, it seems like this use might be one where it’s not only reliable but it appears to outperform existing methods of image compression. Being able to cram more data into less space tends to lead to interesting developments, so I will be keeping my eye on this....
Marko: "Perez won't drive for Red Bull forever" from Motorsport-total.com (lemmy.world)
Holy hell, RB has completely unraveled and fallen apart
Too early to say, but it does seem that the new flexible parts rule clarification could have been aimed at their car
Can you drive a manual transmission?
And where are you from? And how old? Not “do you” but just if you know how....
Introducing Proof-of-Work Defense for Onion Services | Tor Project (blog.torproject.org)
A new DoS protection mechanism for Tor leveraging Proof-of-Work.
[@Autosport] How the standings looked going into the F1 summer break... Last year vs This year (lemmy.world)
Baldur's Gate 3 is a 122GB download and you can't preload any of it (www.pcgamer.com)
Larian has finally confirmed that preloading won't be available, apparently because of a limitation with Steam.
Failed replication of claimed superconductor reported on arxiv (arxiv.org)
Controversial Physicist Faces Mounting Accusations of Scientific Misconduct (www.scientificamerican.com)
Allegations of data fabrication have sparked the retraction of multiple papers from Ranga Dias, a researcher who claimed discovery of a room-temperature superconductor
This Machine Could Keep Moore’s Law on Track (spectrum.ieee.org)
2023 Belgian Grand Prix - [SPRINT SHOOTOUT] discussion thread
FORMULA 1 MSC CRUISES BELGIAN GRAND PRIX 2023...
Musk rushes out new Twitter logo—it’s just an X that someone tweeted at him (arstechnica.com)
deleted_by_author
AI model output quality decreases when trained with AI models (futurism.com)
“Our primary conclusion across all scenarios is that without enough fresh real data in each generation of an autophagous loop, future generative models are doomed to have their quality (precision) or diversity (recall) progressively decrease,” they added. “We term this condition Model Autophagy Disorder (MAD).”...
Ricciardo to replace De Vries at AlphaTauri from the Hungarian Grand Prix (www.formula1.com)
Daniel Ricciardo will make a sensational return to front-line Formula 1 racing at the upcoming Hungarian Grand Prix, with the Australian set to replace Nyck de Vries at the AlphaTauri team for the rest of the season.
Are kbin upvotes/downvotes public?
just wanted to ask this bc lemmy supposedly has public upvotes/downvotes and I don't know if kbin has it too
Tell HN: Nearly all of Evernote’s remaining staff has been laid off | Hacker News (news.ycombinator.com)
Its acquirer (Bending Spoons) has taken over operations. They’ve also hiked subscriptions prices and told customers they intend to use new revenues to pay for new features. How they intend to do that without any staff is something I would like to know about....
[@tgruener] Aston Martin made sure their drivers wouldn't get any penalties before protesting. Also confirmation gravel traps will be fitted outside T9 and T10 at Red Bull Ring with avail from MotoGP. (lemmy.world)
We are launching KBIN fully managed service (elest.io)
Hey dear community, we just launched today our fully managed hosting service of KBIN...
The Lord of the Rings: Gollum studio Daedalic reportedly lays off 25 employees, exiting game development (www.eurogamer.net)
Will it ever be possible for kbin to federate with PeerTube?
I know this is a silly question, especially as I'm currently focused on local-only feeds, but there's just one part of the Fediverse I wish would get more love: PeerTube. While there aren't a lot of people actively looking for alternative video hosts outside the big guys (Twitch, YouTube, Dailymotion, etc.), it would be nice to...
YouTube is testing a more aggressive approach against ad blockers (www.androidpolice.com)
It looks like Google are pushing pretty hard on AdBlockers now. Looks like a pretty aggressive new UI from them....
Lemmy.ml is blocking all requests from /kbin Instances (kbin.social)
I discovered yesterday evening that Lemmy.ml is blocking all inbound ActivityPub requests from /kbin instances. Specifically, a 403 'access denied' is returned when the user agent contains "kbinBot" anywhere in the string. This has been causing a cascade of failures with federation for many server owners, flooding the message...
Kinda strange that people are so crazy about ChatGPT's abilities for bad
There are lots of articles about bad use cases of ChatGPT that Google already provided for decades....