janriemer

@janriemer@floss.social

Software Engineer with an incredible thirst for knowledge, who shares that knowledge with others, so that they can become their best selfs.

Interested in #Rust, #WebAssembly, #TypeScript, #OpenSource, #WebDev, #WebDesign and a lot of other interesting stuff.

Let's have some deep conversations about interesting topics. 🙂

I'm open-minded, but also hold strong opinions.

Dare to think for yourself.

Be kind.

Strive for excellence.

(moved from mastodon.technology - on Mastodon since Feb 2019)

This profile is from a federated server and may be incomplete. Browse more on the original instance.

janriemer, to programming

If you thought had a crazy then you probably haven't heard of yet (by kaleidawave):

Introducing Ezno:
https://kaleidawave.github.io/posts/introducing-ezno/

"In short, it is a featuring checking, and for building full-stack (rendering on the client and server) websites."

Ezno in '23:
https://kaleidawave.github.io/posts/ezno-23/

This type system is craaaazyyy!🤪

Can't wait for 1.0! :awesome: 🚀

janriemer, to random

Rust Rust Rust Rust Rust Rust Rust Rust Rust Rust Rust Rust Rust Rust Rust

:ferris: 🎡 :rust: 🦀

jonhoo, to random
@jonhoo@fosstodon.org avatar

Because scheduling is hard, this week's scheduled stream will be on Saturday, not Friday, at 5pm UTC (https://everytimezone.com/s/0314c2fd). Haven't decided yet what it'll be on, but likely either part 2 of distributed systems challenges, or a Crust of Rust on collection types in std.

janriemer,

@jonhoo Yaaay! :awesome:

Aahhh, both topics sound equally interesting! 🤩 I've seen your little survey below, but I won't vote, because it would basically be a coin-flip for me.😄

Let's see, what the crowd says.🙂

janriemer,

@jonhoo Hm...when I click on the link, I get redirected to:

https://workspace.google.com/products/calendar/?hl=en

Is it, because I'm not signed in to Google (I don't have a Google account)?

janriemer,

@jonhoo Yes, both work. Thank you! 💚

chrisg, to rust
@chrisg@fosstodon.org avatar

Every time i think i've reached a use case for something "smarter" than 's Vec<>, after i dump assembly i realize the folly of my ways.

Maybe i should make a video comparing Vec<> with something else, down to the assembly and run some microbenchmarks.

janriemer,

@chrisg @jntrnr Reusing memory, when you are memory-bound, can give you a significant performance boost, indeed. You not only need to allocate less, but you also need to drop less.

Some might not be aware that there is not only .clear() for clearing the Vec and keeping allocated memory, but there is also drain():

https://doc.rust-lang.org/std/vec/struct.Vec.html#method.drain

It returns an iterator over a certain Range, while removing elements from the Vec and keeping allocated memory in the Vec.

janriemer,

@chrisg @jntrnr If you are interested in a real world use-case for drain(), have a look at csv-diff:

https://gitlab.com/janriemer/csv-diff/-/blob/27b80a9f42183667b431df2b91dacfa4b8f457d1/src/diff_result.rs#L588

It basically allows to process intermediate results and remove elements from the hash map that can already be processed.

For those elements that can't yet be processed, we need to put them back into an intermediate hash map:
https://gitlab.com/janriemer/csv-diff/-/blob/27b80a9f42183667b431df2b91dacfa4b8f457d1/src/diff_result.rs#L600

In the end, we just need to swap intermediate hash map with the actual hash map:
https://gitlab.com/janriemer/csv-diff/-/blob/27b80a9f42183667b431df2b91dacfa4b8f457d1/src/diff_result.rs#L640

janriemer,

@chrisg @jntrnr I think, this change has led to a 35% performance improvement in the best case scenario. 🚀

chrisg, to random
@chrisg@fosstodon.org avatar

Refactoring the codebase to add some functionality, and the thing right now is like a complicated gadget disassembled to pieces and spread all over a desk.

You are sure you heard some pieces falling to the floor, but you're not sure exactly where they landed, You'll find them later.

And you really should have taken some photos during disassembly to know how to put it back together.

It'll probably work out fine.

janriemer,

@chrisg This is such a beautiful and accurate description!

Isn't the feeling during refactoring weird?! It goes from "concrete knowledge" before refactoring to "intuition-based knowledge" during refactoring, trusting that in the end everything will fall into place, until it is at "concrete knowledge" again after everything is done.

It's basically a detour into the unknown - scary and exciting at the same time.

Mara, to rust
@Mara@hachyderm.io avatar

deleted_by_author

    •
    janriemer,

    @Mara This is disgusting!🤮
    I'm sorry you have to experience this.😞

    zkat, to random

    Ok it’s been a real rough day and a real rough week but you know what, I’m a parent now and I’m gonna call that some level of win 🥇

    janriemer,

    @zkat Congratulations, Kat! 🤗

    MrHedmad, to random
    @MrHedmad@genomic.social avatar

    I have a very large .csv file with a numerical matrix saved within. I need to calculate the mean of many selections of values in each row. (e.g. In each row, the mean of the values at index 1, 3, 52, 123; then, in the same line, values 2, 3, 12, 29, 67, etc...)

    My file is HUGE, even in row length (a row has like, 8000+ items), so I need this to be fast. I know the indexes I need to average at the start of the computation, but don't have enough memory to load the whole file at once.

    janriemer,

    @MrHedmad If I interpret your problem correctly, you probably don't want to use Parquet or Arrow for this (as others have suggested), because your operation is fundamentaly row-based and not column-based.

    You can use csv crate in a "streaming" fashion by reusing your StringRecord in every iteration of row parsing:

    https://docs.rs/csv/latest/csv/struct.Reader.html#method.read_record

    Something like this:
    https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=3f6ba982764f8e97a20bf718d4546f25

    It will basically only require memory for one row at a time.

    1/2

    janriemer,

    @MrHedmad If you want to go even further, you might be able to split your file into chunks and have threads operate on those chunks individually. However, this will be more complicated, because how will you determine the correct boundary on where to split without resulting in invalid csv?

    Here is a very interesting discussion in the Rust forum about this (with a normal txt file, though, and not csv)

    https://users.rust-lang.org/t/reading-a-file-4x-faster-using-4-threads-works-threaded-is-faster/41180

    2/2

    janriemer, to rust

    Whoop! 🎉 I've just called a module from my app! :awesome:

    It uses rsw with it's Vite plugin:

    https://github.com/rwasm/vite-plugin-rsw

    Good examples can be found here:
    https://github.com/rwasm/learn-wasm

    It requires a bit of config, but not too bad, given that it's bleeding edge.

    What tripped me up in the very end:
    You need to call init() first from your wasm module, otherwise error "wasm is undefined", when calling your function.

    https://users.rust-lang.org/t/wasm-pack-without-node-js-npm/55949

    janriemer, (edited ) to random

    Interview with a ChatGPT-4 Enthusiast

    https://farside.link/https://www.youtube.com/watch?v=OWIxzE2D7Xk

    (or YT: https://www.youtube.com/watch?v=OWIxzE2D7Xk)

    Haha, this is so funny!😅 😂

    How can they have a straight face while filming this!?😄

    janriemer, (edited )

    ...and another one:

    Interview with a Twitter Futurist

    https://farside.link/https://www.youtube.com/watch?v=sxdDVAv6QDY

    (or YT: https://www.youtube.com/watch?v=sxdDVAv6QDY)

    janriemer,

    @silmathoron Ohh, this is very neat, thank you! ❤️

    I've updated the links.

    janriemer, to rust

    Obscure Rust: reborrowing is a half-baked feature - by Thoughts with wings (2022-06)

    https://haibane-tenshi.github.io/rust-reborrowing/

    appelgriebsch, to random
    @appelgriebsch@fosstodon.org avatar

    With more than 2 years working on this project.... and lot of learnings for deploying software on embedded devices.... happy to see it hitting various markets and getting adopted...
    https://www.youtube.com/watch?v=KPjgOUcytCE

    janriemer,

    @appelgriebsch Congrats, Andreas!🎉

    Looks really promising. ☘️

    Is there any data yet about the avg. savings in use of plant protection with this method? Of course, it strongly depends on the number of plants per m³ (more savings, when plants are relatively far apart from each other, I guess!?) and probably a lot of other variables.
    Just curious. 🙂

    janriemer,

    @appelgriebsch Wow, 60% is a lot, nice! 👍

    😄 I've already assumed that it is powered by Rust. 😉 :ferris: Very cool! 👍

    janriemer, to random

    😯Wait, what!? is not growing exponentially, but has reached it's plateau!?

    How can that be!? This is such a surprise! Oh my!😱

    /s

    OpenAI’s CEO Says the Age of Giant AI Models Is Already Over

    https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/

    janriemer,

    @alcinnz Yes, exactly. The irony is:

    • the internet is full of content claiming AI is going to grow exponentially

    • AI is learning from text on the internet

    Well, seems like AI has a high quality knowledge base right there... /sarcasm

    janriemer,

    @chiefgyk3d
    Yes, you're right. More specialized will probably mean better quality, but the current underlying architecture of AI is still flawed.

    We still don't know, how these systems work. They are black boxes.
    Given the same input, they produce different outputs, which basically makes them untestable:

    Unpredictable Black Boxes are Terrible Interfaces

    https://magrawala.substack.com/p/unpredictable-black-boxes-are-terrible

    @alcinnz

    janriemer, (edited )

    @danieldekay What makes you think improvements in AI were exponential? And on what metric exactly?

    chrisg, to programming
    @chrisg@fosstodon.org avatar

    I have picture lock 🙂

    Editing timeline for the next video. Some clip names give up the topic i cover.

    This one was heavy on research and i learned a bunch of stuff. I think it will be useful to many.

    And yeah, it's all

    janriemer,

    @chrisg Hmhmhmhmhm.... 🤔

    I see a bunch of try's in there...you're not coincidentally making a video about the try trait!? :awesome: :awesome: :ferris:

    janriemer,

    @chrisg Nice! That sounds super interesting. Definitely gonna watch it. :awesome: 👀

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • JUstTest
  • tacticalgear
  • DreamBathrooms
  • InstantRegret
  • magazineikmin
  • Youngstown
  • everett
  • anitta
  • slotface
  • GTA5RPClips
  • rosin
  • thenastyranch
  • kavyap
  • mdbf
  • Leos
  • modclub
  • osvaldo12
  • Durango
  • khanakhh
  • provamag3
  • cisconetworking
  • ngwrru68w68
  • cubers
  • tester
  • ethstaker
  • megavids
  • normalnudes
  • lostlight
  • All magazines
    •