malcircuit,
@malcircuit@thingy.social avatar

The reality of using SSDs:

Copy 100 GB directory of RAW images: 5-10 minutes

Copy 90 GB folder with ~200k files, many of which are tiny: 5-500 days (estimated, and constantly changing) :blobcateyes: :blobsweat:

malcircuit,
@malcircuit@thingy.social avatar

It's been running for 1.7 hours so far and copied 3.3% of the files (0% of the total bytes) and I'm not really sure what I'm supposed to do here.

I kind of need this drive to do stuff... sometime in the next year, please lol

malcircuit,
@malcircuit@thingy.social avatar

If you weren't aware, unless precautions are taken, SSD performance can tank in situations where you're doing stuff with a lot of very small (few KB) files. I don't remember the exact technical reason for it. Something to do with the size of the sectors of memory being 4 KB and wear leveling. I learned about it like a decade ago when SSDs were a new and shiny technology and promptly forgot lol

qlp,
@qlp@linh.social avatar

@malcircuit Yeah, it's a combination of file system block size, NAND page size and the controller trying to combine writes to reduce write amplification.

There can also be some OS-level writing caching or file system tuning that can also make things a bit worse.

malcircuit,
@malcircuit@thingy.social avatar

@qlp Oh yeah, now I remember. The drive controller packs the smaller files into a single block, so if you do something like delete or modify one of the files, the drive has to copy the entire block into RAM to manipulate it, and it writes to a different block for wear leveling. So a "simple" change actually ends up being multiple (relatively) computationally and time intensive steps. Moving an entire block unmodified is easier, which is why large file transfers on SSDs are insanely fast.

qlp,
@qlp@linh.social avatar

@malcircuit Yeah, since there is an ever decreasing amount of program/erase cycles for NAND cells (it gets worse when we try to pack more bits per cell and reduce the cell size).

The issue is exacerbated on the lower-end of the SSD spectrum where the NAND controllers do not have DRAM caching. It's the same reason why USB thumb drives and SD cards can have some really bad random I/O or write performance versus older SATA SSDs with DRAM caches.

malcircuit,
@malcircuit@thingy.social avatar

BTW, that "dumb" hard drive or SSD is actually a full-blown computer in its own right.

The motherboard? That's a computer too. Probably actually many CPUs on any given motherboard.

Intel and AMD processors have microcode-based instructions, which basically means some x86 CPU instructions are actually small software programs that run inside of a special CPU inside of the CPU so you can compute while you compute.

It's crazy to think about how many CPUs are inside a modern computer.

qlp,
@qlp@linh.social avatar

@malcircuit Intel Manageability/Management Engine includes a tiny Intel Quark core that runs a version of MINIX, if I remember correctly.

Many wireless radio controllers run their own OS to allow for software-defined radio control. Servers with lights-out management have at least one full computer on a package or two. The MacBook Pro Touch Bar is a mini-computer (T1 running a customized version of Watch OS, I think?) controlling it.

Computers (or holograms) all the way down 😂

  • All
  • Subscribed
  • Moderated
  • Favorites
  • random
  • kavyap
  • thenastyranch
  • GTA5RPClips
  • tester
  • InstantRegret
  • DreamBathrooms
  • ngwrru68w68
  • magazineikmin
  • everett
  • Youngstown
  • mdbf
  • slotface
  • rosin
  • cisconetworking
  • megavids
  • khanakhh
  • normalnudes
  • osvaldo12
  • cubers
  • tacticalgear
  • Durango
  • ethstaker
  • modclub
  • anitta
  • provamag3
  • Leos
  • JUstTest
  • lostlight
  • All magazines