datamaps

@datamaps@social.linux.pizza

#data professional, mostly #analytics engineer and some #dataOps, using #foss like #rstats, #shiny and #bash to build #dataviz and #geospatial Web Apps on #linux #ubuntu servers.

here and there also some elem #programming with #clang #fortran and #javascript

every now and then tinkering with #arduino and #raspberrypi

addicted to crime TV series, my timeline of fame:
NYPD Blue > Sopranos > Shield/Wire > Breaking Bad > Southland.
lately also nordic noir on @WalterPresents

#fedi22 #nobot

This profile is from a federated server and may be incomplete. Browse more on the original instance.

smach, to rstats
@smach@masto.machlis.com avatar

“Let’s be honest – all of your R Shiny app logic is contained in a single app.R file, and it’s probably several hundreds of lines long.” 😅 (Not always, but more often than I’d like to admit)
How to Create Your First R Shiny Module, by Dario Radečić on the @appsilon blog
https://appsilon.com/r-shiny-modules/
@rstats

datamaps,

@analyst42 @smach @appsilon @rstats

having multiple files or using modules are two different things though, as files organize code, while module organize logic.

In any case, even wneh not using modules, you're not restricted to a fixed number of files, as you can use, as I often do, ui.R and server.R as simple indices to source other files that are the actual containers of the app code. My most complex app to date has a total of 17 files, divided by tabs and sections for most complex tabs.

datamaps,

@analyst42 @smach @appsilon @rstats

and if you encapsulate (most of) your data and logic (in the form of "normal" functions or modules, whatever you prefer) in a dedicated package you can often use a single very short app.r even if the logic is complex, meanwhile facilitating maintenance and distribution

ramikrispin, to datascience
@ramikrispin@mstdn.social avatar

TIL that there is an R version of the polars Python package 👇🏼

Code ➡️: https://github.com/pola-rs/r-polars
Docs ➡️: https://rpolars.github.io/

datamaps,

@ramikrispin

I've been kind of willing a few times to learn it but there are so many breaking changes at every subversion that I'm not that interested to use it

what's in it anyway that should push me to supersede data.table?
(I know, there currently is a bit of commotion around its future, but let's put it aside for the moment being)

_wurli, to random
@_wurli@fosstodon.org avatar

Genuine question: are there any tools outside of that come close to dplyr + tidyr + purrr for data wrangling?

datamaps,

@JorisMeys @_wurli

I would add {collapse} which is a promising alternative to {data.table} (see {sf} polygon binding) https://sebkrantz.github.io/collapse/

In fact, I would never have thought anyone would take the tidyverse for a benchmark comparison in data engineering LOL slow, lots of dependencies, anti-reprex

datamaps,

@_wurli @JorisMeys

can you beat something like
dt[, sum(V1), V2]
for summing V1 grouping by V2?
the time it takes you to do the same in dplyr I've already written the report ;-)
simplicity is very subjective, the verbosity of the TV is something I don't like at all

and besides speed, it's also important that I can run the same code years later with no hassle, instead of debugging which commands have been deprecated over the years, dplyr has actually become quite the tool to avoid

datamaps,

@_wurli @JorisMeys

yeah, I know that issue, it's dangling like a ghost sword over my head for a while, but as far as I'm concerned I'm OK with DT staying like it is (apart from bugs), I add new things I need building my own functions, that's what programmers do isn't it?

on the other side, yes, base can do all you need, possibly less efficiently, not sure for example why so many people disregard so much the *apply family... where's the diff with purrr things?

datamaps, to random

this is what happens when you get binmen angry 🤔

Elendol, to random
@Elendol@hachyderm.io avatar

Funny thing I just found out this week after living 3 years at the same place: I have a bus stop to the city centre much closer than the tramway stop. Like 15 minutes closer. During peak hour the tram is still better because it cuts through the city and traffic, but any other time the bus will work fine (albeit we are talking about Dublin bus, so we can't put too much trust into them...)

datamaps,

@Elendol

a few years ago I use to work in an office near St Pauls in London, and for a while I used to walk the 35-40 mins needed (~2mi). There were these roadworks over London Bridge, that the bus from my house needs to cross, and one hour was then the normal instead of the usual 30. Afterwards I was so used to walk that I didn't bother with the bus if it wasn't raining that much

tk, to random
@tk@bbs.kawa-kun.com avatar
datamaps,

@tk

that's not completely true, as the chip has not been inserted into the cheese, but into the labels on the rind of wheels. they are edible only for additional safety as it's highly unlikely that any consumer will ingest one.

https://www.geopop.it/come-funzionano-i-chip-commestibili-del-parmigiano-reggiano-per-garantirne-lautenticita/

ramikrispin, to datascience
@ramikrispin@mstdn.social avatar

My R Shinylive app is now on Github Pages. It was simple and straightforward to deploy the app on Pages, a tutorial to follow...

Please be patient while loading 😝

Pros:

  • Serverless
  • Easy to deploy as a website on Github Pages (and similar)

Cons:

  • Slow load time
  • Still experimental

https://ramikrispin.github.io/shinylive-r/

datamaps,

@ramikrispin

there's something I don't get though, possibly because I still have to dig deeper into the concept

AFAIK with webr the loading time is (quite) long the first time because of the need to download the R bytecode and the required packages, but afterwards it should be nearly instantaneous. with your app it always take 40-50 secs

Considering the simplicity of the app there is no way anyone would do that (besides pasttime) as in JS is possibly much simpler to build and surely faster

crawfordsm, to python
@crawfordsm@mastodon.social avatar

What are your favorite introductions or tutorials to programming for scientists?

This could be for a specific language or version control or to the command line.

#OpenScience #OpenSourceSoftware
#python
#RStats

datamaps,

@crawfordsm

hands down
https://www.youtube.com/playlist?list=PLhQjrBD2T380F_inVRXMIHCqLaNUd7bN4

ojala, to random
@ojala@mastodon.nz avatar

deleted_by_author

    •
    datamaps,

    @ojala
    I find a chromebook (not the cheapest one) much easier and convenient to use with RSS than a tablet, you don't really need any proc power, and a true backlit keys is a must. I had an asus flip C302CA that last me 5 years with no problems

    datamaps,

    @ojala @aligorith

    I went the other way, I've started a couple of years ago using a VPN with dedicated IP to have an easy way to secure access, through nginx server blocks denying all IPs but mine, on top of usr/pwd.

    only when you're the admin of your box you realize how many 100s, if not 1Ks, of people (bots) try to access it on a daily basis

    datamaps, to random

    A quick comparison between over the internet and locally vs data.table

    https://carbon.now.sh/WUTpkE9SVAoSJPJchIF7

    ERDonnachie, to random

    So, I'm using 2023.06.1 and it can't find it's own installation of pandoc.

    RStudio ignores any environment variable set in the Renviron file, or system-wide, and always sets RSTUDIO_PANDOC to "c:\Program Files\RStudio\pandoc\bin" (i.e. the old location, which I don't have defined anywhere). Consequently, rmarkdown::pandoc_available() is FALSE.

    I have no idea why it is doing this. Any ideas?

    @Posit

    datamaps,

    @ERDonnachie @Posit
    have you tried to set it through the proper windows environment variable panel ?

    datamaps, to random

    it's possible not many people know about a .table trick with loading a zipped csv table downloaded from the internet, so I let it here.

    datamaps,

    @stevensanderson

    I'm not sure if unlink is actually of any use, AFAIK it does not delete anything, it just flags the file for deletion, but (at least on a linux machine) everything in the /tmp dir (that's where they exist) is always deleted at reboot, or after 10 days from creation

    brodriguesco, to python
    @brodriguesco@fosstodon.org avatar

    My 2 cents regarding in

    datamaps,

    @brodriguesco
    Mate, that's more like 1$ than 2¢ 🤣

    Anyway, i wouldn't bother, this is more like a M$ marketing tactic (like github) to avoid hemorrhaging office users, that at the moment are buying it just for how cheap onedrive is. And the more you talk, even in negative, the more you're just doing them a favour

    DataAngler, to random
    @DataAngler@vis.social avatar

    Going to present to my office on adopting research practices for reproducibility...do I dare try to learn {docker} 📦 ? It seems critical for ensuring someone else can reproduce the analyses.

    datamaps,

    @DataAngler

    besides repr, docker is also a (easier?) way to scaling up shiny apps, in the world, and more generally, for faster deployment, maintenance, and migrations, and because of env isolation, for better security.

    bullivant, (edited ) to UKpolitics

    So we are about to lose access to WhatsApp, Apple and Facebook in the UK. If it happens I cannot imagine what the reaction of the public will be [irony]?

    Seriously what on earth are the Tories thinking? They are as reliant on these platforms as the rest of us. Do they think that the tech companies are bluffing and that the UK is too important a market to exit? If they do, they could be in for an awful shock.

    https://www.bbc.co.uk/news/technology-66256081

    datamaps,

    @bullivant

    Not the same issue, but couple of months ago the Italian data protection agency "banned" chatgpt over breach of privacy rules and age checks, reactivating it a month later after OpenAI addressed the issues

    Soon afterwards the EU started to work on its own set of rules

    So, no, like there's no "too big to fail" there's also no "too small to notice", bollocks to let big corp go on with status quo

    Small changes can lead to big ones, while localized ones can lead to global ones

    hadleywickham, to random
    @hadleywickham@fosstodon.org avatar

    If you're interested in the health of the community, a few stats for you: in the last week, RStudio was opened 3.5 million times (i.e. we logged that many checks for an update); in the last month, ~650k people visited a tidyverse website and ~65k people used posit.cloud; in the last year, >2 billion packages were downloaded.

    datamaps,

    @jimbob @hadleywickham
    meanwhile both and data "fans" think that SAS and SPSS are both dead, but they have an estimated global market value of resp 3+ and 6+ billions, go figure.

    Edit: Actually the $3B for SAS Institute are revenues being it a private company.

    Private
    datamaps,

    @RUGatHDSI @rstats

    I started watching it but stopped when I heard that the *apply family are legacy and should be avoided to incorporate those functions into new code?!?!? LOL

    johnmackintosh, to random
    @johnmackintosh@fosstodon.org avatar

    I got asked yesterday if there was a "best" way to add a 'totals' row to an existing dataframe.

    Aside from {janitor}, a lot of the answers on Stack Overflow are pretty convoluted.

    I'm sure I'm not the first person to come up with this solution, in fact there was a very similar one on SO, only that was a one liner and coerced existing numerics to character

    https://gist.github.com/johnmackintosh/c0dc876a09e0e6949bd4c1e9e41e99cf

    datamaps,

    @johnmackintosh

    your one-liner for data.table:

    rbindlist(list(y, y[, lapply(.SD, sum), .SDcols = which(y[, lapply(.SD, is.numeric)] == TRUE)]), use.names = TRUE, fill = TRUE)

    Besides, IMHO totals should be added as a row only to tables (on reports) not dataframes (for analysis)

    brodriguesco, to datascience
    @brodriguesco@fosstodon.org avatar

    I’m giving a presentation on how to make / reproducible using on 𝟭𝟵 𝗝𝘂𝗹𝘆 for , https://rb.gy/21920

    it’s based on my book "Building reproducible analytical pipelines with (https://raps-with-r.dev/)

    I’ll quickly walk you through how you can use {renv}, {targets} and to make your project reproducible!

    datamaps,

    @brodriguesco @transportationtalk

    I'm not sure your DF ensure repr. To stay on a safer side you should lock the OS libraries as well.I've been a few times in a situation when an old R package does install with a newer library than expected, making most times rewriting the R code the better solution

    Also, while you can choose the R version, the OS version from the rocker build can be different from the one you used for the project

    that's a couple of reasons I always start from scratch

    datamaps,

    @brodriguesco @transportationtalk
    no worries ;-)
    what I'm saying is if you can write down and manage a dockerfile why you want to start with a rocker image anyway? they're handy to build solutions quickly, but not safe for complete repr

    IMHO even running
    apt update && apt upgrade
    soon after downloading the image is not really safe

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • JUstTest
  • cubers
  • DreamBathrooms
  • InstantRegret
  • tacticalgear
  • magazineikmin
  • Youngstown
  • thenastyranch
  • mdbf
  • slotface
  • rosin
  • modclub
  • kavyap
  • ethstaker
  • provamag3
  • osvaldo12
  • khanakhh
  • cisconetworking
  • Durango
  • everett
  • ngwrru68w68
  • Leos
  • normalnudes
  • GTA5RPClips
  • tester
  • megavids
  • anitta
  • lostlight
  • All magazines
    •