@jmcastagnetto@mastodon.social
@jmcastagnetto@mastodon.social avatar

jmcastagnetto

@jmcastagnetto@mastodon.social

Chemist (PhD, NYU '97), Bioinformatics mangler, Biomedical Informatics dabbler, Data artisan, R scribbler, Metalloprotein DB creator, former PHP/PEAR kludger.
#Rstats #OpenData #Science #BiomedicalInformatics #DataScience #DataVisualization #DataEngineering #PublicHealthInformatics #DataForGood
#dator

This profile is from a federated server and may be incomplete. Browse more on the original instance.

hrbrmstr, to random
@hrbrmstr@mastodon.social avatar

ZOMGOSH THEY RESURRECTED ARCHIE https://archie.serialport.org/

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@hrbrmstr

Veronica where are thou? :-)

jmcastagnetto, to ai
@jmcastagnetto@mastodon.social avatar

A report from Microsoft & LinkedIn, about at , indicating the rise in use of generative AI for work tasks.

"AI at Work Is Here. Now Comes the Hard Part" https://www.microsoft.com/en-us/worklab/work-trend-index/ai-at-work-is-here-now-comes-the-hard-part/

jmcastagnetto, to random
@jmcastagnetto@mastodon.social avatar

@coolbutuseless -- just saw this on the incognito site :-)

https://x.com/JosiahParry/status/1787871177677115698

{yyjsonr} rewlz in speed and size

jmcastagnetto, to llm
@jmcastagnetto@mastodon.social avatar

"Researchers at# Stanford Introduce : A Formal Query Language for Integrating Structured and Unstructured "

https://www.marktechpost.com/2024/05/04/researchers-at-stanford-introduce-suql-a-formal-query-language-for-integrating-structured-and-unstructured-data/

See paper at: https://arxiv.org/abs/2311.09818, and code at: https://github.com/stanford-oval/suql

hrbrmstr, to random
@hrbrmstr@mastodon.social avatar

@todb did y'all vet https://hiddenlayer.com/research/r-bitrary-code-execution/ ? (it's kind of "expected functionalityโ€)

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@hrbrmstr @todb

Now they need to coin a catchy moniker like "RDS injection" or something like that ๐Ÿ˜‰

Cmastication, to random
@Cmastication@mastodon.social avatar

I bought a used foot pedal for my Mac. It didn't have software for a Mac available and it does not show up as a keyboard or mouse. So I had to learn how to read usb input using python. Fun little yak shaving event. I have it plumbed up to turn speech to text on and off. Kinda fun.

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@Cmastication Very cool, for a minute there I thought it was one of those emacs foot pedal contraptions :-)

smach, to rstats
@smach@masto.machlis.com avatar

File import/export in R is simple and elegant with the {rio} ๐Ÿ“ฆ. It uses just 2 main functions for dozens of file types: import() and export(). Whether .zip, .xlsx, Google sheets, json, .rds, .csv or more, rio handles file-extension checks and selecting the right functions.
http://gesistsa.github.io/rio/
There's also a convert() function.
One of my favorite R packages!
By Thomas J. Leeper, Chung-hong Chan, David Schoch & Jason Becker
@rstats

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@smach @rstats

In my experience, {rio} has trouble with malformed CSV files, often truncating data when it encounters a different number of columns in a file. I find that {vroom} and {readr} (which uses {vroom}) are more robust with pathological CSVs.

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@schochastics @smach @rstats @chainsawriot

I had to look up my notes from last year's class, and using that I've made an example that shows the problem. You can see it at: https://gist.github.com/jmcastagnetto/cf25a604b564ac15b9beb43f7cc1cb91

We were looking at the data for using , and that is when rio::import() failed after reading only 87,872 records out of the expected 501,692 -- readr::read_csv() worked, reading the whole thing w/ a warning.

HTH

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@chainsawriot @schochastics @smach @rstats

(1)
Thanks for your comments in the gist. Most of what you mention I had to explain to participants who deal with non-ideal CSV (and XLSX) files daily. Fortunately I've been using R since 2003, so I had experience w/ some of its edges.

Initially, we picked {rio} to make things simpler for total beginners, but because it needs very clean and regular input, it caused more confusion when they tested with their real-life data.

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@chainsawriot @schochastics @smach @rstats

(2)
I can appreciate the uniformity in API that {rio} provides, but I am not sure if it is appropriate for teaching R to people starting in programming (and R), who are using real life data (which can be pathological). They will be more confused than cognizant of what is going on, and will not have the tools, knowledge and experience to fix the issues and read the data correctly.

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@schochastics @smach @rstats @chainsawriot

Yep, and that is just an issue we knew from this dataset. When we tried using {rio} to teach beginners in R to read input data, using their own real life data, there were even more issues, and different ones. So we had to explain what was going on, that sidetracked the aim of the session: how to read data.

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@chainsawriot @schochastics @smach @rstats

I've put some more context (most already mentioned in this thread), in the gist:

https://gist.github.com/jmcastagnetto/cf25a604b564ac15b9beb43f7cc1cb91

Thanks everyone for your input

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@smach @schochastics @rstats @chainsawriot

Yes, I agree with that approach, giving them a taste of what can be done, then working with them to get there.

In our case, we show reports, tables, graphs that are of use for health surveillance, then we go step by step in each session on each aspect: reading data, exploring it, cleaning it, making tables, calculating statistics, making charts, etc., and we try to show some of "warts" in the data along the way, so it is more tangible to them.

hrbrmstr, to random
@hrbrmstr@mastodon.social avatar

Drop (2024-04-22): Monday Morning Grab Bag

Today's Drop discusses three spiffy tools: Duckling, neko, and tv. Duckling is a data management app for viewing and querying multiple data sources. Neko is a remote desktop tool for private web browsing and collaborative projects, while tv is a command-line tool for pretty-printing CSV files. Each tool offers unique features for data enthusiasts and remote system users.

http://dailydrop.hrbrmstr.dev/2024/04/22/drop-455-2024-04-22-monday-morning-grab-bag/

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@hrbrmstr All three tools look very cool, 'tv' in particular will be useful for CLI / remote work.

Speaking of data tools, have you used the 'nu' shell (https://www.nushell.sh/), where every command can generate the equivalent to a dataframe. I am not sure if you had mentioned in another drop before.

jmcastagnetto, to vscode
@jmcastagnetto@mastodon.social avatar

"Researchers Observed Visual Studio Code Extensions Stealing Users' "

https://gbhackers.com/researchers-observed-visual-studio/

'... ReversingLabs has uncovered a series of VS Code extensions that designed to siphon off sensitive information from unsuspecting users ...'

hrbrmstr, to javascript
@hrbrmstr@mastodon.social avatar

2024 Day 03: Makeover

Went back into the blog archives (2017!) to do a modern makeover of my Maine ๐Ÿฆž landings connected dot plot.

PDF data wrangling all done in .

https://observablehq.com/@hrbrmstr/2024-30-day-chart-challenge-day-03-makeover

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@hrbrmstr

That looks like a "drunken walk" rather than a random walk

hrbrmstr, to random
@hrbrmstr@mastodon.social avatar

Bonus Drop (2024-03-24): Method To The Madness

Today's Drop discusses 3 resources, madonctl, csvlens, and DuckDB, and how they can be combined for data FUN. madonctl is a Mastodon client for the terminal, csvlens is a CLI tool for viewing CSV files, and DuckDB is a powerful database system that is quite magical.

http://dailydrop.hrbrmstr.dev/2024/03/24/bonus-drop-46-2024-03-24-method-to-the-madness/

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@hrbrmstr Did not know about csvlens, a very cool tool indeed.
I've been using visidata (https://www.visidata.org/) for command line dataset exploration/quick mangling, now I got another tool to play with :-)

hrbrmstr, to random
@hrbrmstr@mastodon.social avatar

Bonus Drop #45 (2024-03-17): Command & Conquer: JavaScript Edition

Today's Drop discusses the use of Thorium and mprocs for efficient web development, as well as the VS Code remote debugging experience. It emphasizes the importance of using the right tools for productivity and enjoyment. It also provides examples of configurations for different types of projects in VS Code. These tools and practices aim to improve the development process.

http://dailydrop.hrbrmstr.dev/2024/03/17/bonus-drop-45-2024-03-17-command-conquer-javascript-edition/

jmcastagnetto, (edited )
@jmcastagnetto@mastodon.social avatar

@hrbrmstr The other day I was looking at info on this same browser, and some random comments pointed to a controversy about an image.
Check the Dec 14 changes (https://github.com/Alex313031/thorium/commit/9947421f1480bd1a59b7f79d7a136cce01b3ffc9), and then go to the prev version to see what the controversy was about.
This has been fixed, before that it was NSFW or use by minors. Just an FYI.

jmcastagnetto, to postgres
@jmcastagnetto@mastodon.social avatar

"New startup from creator puts the at heart of stack" (TechCrunch)

https://techcrunch.com/2024/03/12/new-startup-from-postgres-creator-puts-the-database-at-heart-of-software-stack/

hrbrmstr, to random
@hrbrmstr@mastodon.social avatar

Drop (2024-03-12): Typography Tuesday

Three fontastic resources await intrepid Drop readers! The Canva Engineering Blog discusses the complexities and security concerns of font processing software and formats, highlighting vulnerabilities like CVE-2024-25081 and the importance of tools like OpenType-Sanitizer for protection. Fontpreview is a command-line tool for quickly previewing fonts, while UNCUT.wtf is a free typeface catalogue featuring 152 contemporaryโ€ฆ

http://dailydrop.hrbrmstr.dev/2024/03/12/drop-425-2024-03-12-typography-tuesday/

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@hrbrmstr

Most likely you have already seen this attack vector, but just in case you didn't:

"Using the wrong font could be a major security problem โ€” and possibly not for the reason you might think"
https://www.techradar.com/pro/security/using-the-wrong-font-could-be-a-major-security-problem-and-possibly-not-for-the-reason-you-might-think

Cmastication, to random
@Cmastication@mastodon.social avatar

Nothing harkens back to the fall of Radio Shack like the Cue Cat. Though I argue it was the father of the QR code.

From: @spacerog
https://mastodon.social/@spacerog/111727245707546434

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@Cmastication @spacerog

I remember going to get one because of the novelty of the thing (back there I was in San Diego, CA), but they had run out of them. Big disappointment, because it looked so cool with the PS/2 connector :-)

jmcastagnetto, to postgres
@jmcastagnetto@mastodon.social avatar

" pioneer promises to upend the once more" (The Register)

https://www.theregister.com/2023/12/26/michael_stonebraker_feature/

hrbrmstr, to random
@hrbrmstr@mastodon.social avatar

Beginning the prep for the feast later today.

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@hrbrmstr

Is that for a whole batallion of coders? :-) Merry Xmas!

Cmastication, to random
@Cmastication@mastodon.social avatar

I ran my mouth at posit::conf(2023) and since I was on stage we call it a โ€œkeynoteโ€. Itโ€™s like shitposting only verbal and recorded for posterity so I can explain to my grandkids what I was thinking about in 2023.

https://youtu.be/Pa1PNfoOp-I?si=ikVnlAxCspAFOH9L

jmcastagnetto,
@jmcastagnetto@mastodon.social avatar

@Cmastication

Got to use that equivalence and great definition for "keynote" :-)

jmcastagnetto, to til
@jmcastagnetto@mastodon.social avatar

about "cotp" (https://github.com/replydev/cotp) a tool that works as a / authenticator, and supports importing from (Android).
And is written in

  • All
  • Subscribed
  • Moderated
  • Favorites
  • โ€ข
  • Leos
  • rosin
  • InstantRegret
  • ethstaker
  • DreamBathrooms
  • mdbf
  • magazineikmin
  • thenastyranch
  • Youngstown
  • tacticalgear
  • slotface
  • Durango
  • khanakhh
  • kavyap
  • megavids
  • everett
  • vwfavf
  • normalnudes
  • osvaldo12
  • cubers
  • GTA5RPClips
  • cisconetworking
  • ngwrru68w68
  • anitta
  • provamag3
  • tester
  • modclub
  • JUstTest
  • All magazines
    •