rmaloley,

In front of me: A CSV file with 1.3 million rows.
The problem: Excel/Calc have a row limit of < 1.3M.
Looking for solutions that can help analyze the data.
Data: Microsoft 365 audit log data.

slink,
@slink@fosstodon.org avatar
blacktraffic,

@rmaloley I do all my analysis in Perl/python because I had this issue in 1997 except it was 65k lines in those days.

rmaloley,

@blacktraffic Honestly I wonder if the first step would be to isolate the records for the specific user in question via grep first. Then use that smaller dataset to provide seed data for further analysis on the larger dataset.

blacktraffic,

@rmaloley probably a good start!

rmaloley,

@blacktraffic

> $ grep -i "$user" $data | wc -l
3460

Not a bad start!

blacktraffic,

@rmaloley I have a particular horror of excel because it does tend to force random things to be dates, but it can be handy for quick visualisations.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • programming
  • ethstaker
  • DreamBathrooms
  • cubers
  • osvaldo12
  • mdbf
  • magazineikmin
  • normalnudes
  • InstantRegret
  • rosin
  • Youngstown
  • slotface
  • khanakhh
  • kavyap
  • ngwrru68w68
  • JUstTest
  • everett
  • cisconetworking
  • tacticalgear
  • anitta
  • thenastyranch
  • Durango
  • tester
  • GTA5RPClips
  • modclub
  • megavids
  • provamag3
  • Leos
  • lostlight
  • All magazines