ColinTheMathmo, (edited )
@ColinTheMathmo@mathstodon.xyz avatar

EDIT: Problem diagnosed, a plan has been formulated. Further suggestions and comments welcome, but please read the thread first.


Time for today's adventure in understanding why things don't work the way I think they should.

Please feel free to boost this for reach, or ignore it if you have no interest, and think none of your followers can help.

Today ... a USB disk mounted on an Ubuntu machine.

1/n

ColinTheMathmo, (edited )
@ColinTheMathmo@mathstodon.xyz avatar

EDIT: Problem diagnosed, a plan has been formulated. Further suggestions and comments welcome, but please read the thread first.


So here is a command and its output (names changed to protect the ignorant):

$ df -h /media/user/DiskName

Filesystem Size Used Avail
Use% Mounted on

/dev/sdb1 4.6T 1.7T 2.7T
40% /media/user/DiskName

So it's a 4.6T disk, currently it's 40% used, and has 2.7T available.

Next ...

2/n

ColinTheMathmo, (edited )
@ColinTheMathmo@mathstodon.xyz avatar

EDIT: Problem diagnosed, a plan has been formulated. Further suggestions and comments welcome, but please read the thread first.


Let's "cd" to the disk and create a file:

$ cd //media/user/DiskName
$ touch TestFile.txt

touch: cannot touch 'TestFile.txt': No space left on device

Hmm. That's ... surprising.

3/n

dougmerritt,
@dougmerritt@mathstodon.xyz avatar

@ColinTheMathmo
Perhaps it's out of inodes, which for most filesystems, is a separate pool than data blocks.

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@dougmerritt Ooh ... that might be a problem.

It's a backup disk, and it has a lot of hardlinks. That might have some effect.

Is there a way I can check on the inode question?

dougmerritt,
@dougmerritt@mathstodon.xyz avatar

@ColinTheMathmo
df -i filesystem

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@dougmerritt And there we have it:

$ df -i /media/User/DiskName

Filesystem Inodes IUsed IFree
IUse% Mounted on

/dev/sdb1 152616960 152616960
0 100% /media/User/DiskName

ColinTheMathmo, (edited )
@ColinTheMathmo@mathstodon.xyz avatar

EDIT: Problem diagnosed, a plan has been formulated. Further suggestions and comments welcome, but please read the thread first.


I think it's an EXT4 disk on an elderly 32-bit Ubuntu system.

Is the OS unable to deal with disks this big?

If so, why did it format the disk for me instead of complaining?

Or is it something else.

All assistance welcome, and thank you in advance.

4/n, n=4

penguin42,
@penguin42@mastodon.org.uk avatar

@ColinTheMathmo can you try a 'df -i' and also an lsblk ?

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@penguin42 When I do lsblk it says it's not a block device.

$ lsblk /media/User/DiskName

lsblk: /media/User/DiskName/: not a block device

??

penguin42,
@penguin42@mastodon.org.uk avatar

@ColinTheMathmo Indeed it's not, the /dev/sdc1 or whatever is the block device.

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@penguin42 Ah ...

$ lsblk /dev/sdb1
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb1 8:17 0 4.6T 0 part /media/User/DiskName/

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar
ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@penguin42 @dougmerritt

So ... when I have a hardlink between FileA and FileB ... how many inodes does that use?

penguin42,
@penguin42@mastodon.org.uk avatar

@ColinTheMathmo @dougmerritt A hardlink takes up an inode, just like a normal file.

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@penguin42 OK, so the structure of my backup system will be killing me.

sigh

Right ... need to re-work that.

Thank you.

CC: @dougmerritt

dougmerritt,
@dougmerritt@mathstodon.xyz avatar

@ColinTheMathmo @penguin42
Pretending you were addressing me, you're welcome.

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@dougmerritt Certainly you were the target for some of the thanks.

So:

👍

CC: @penguin42

penguin42,
@penguin42@mastodon.org.uk avatar

@ColinTheMathmo @dougmerritt You can manually specify the number of inodes on a device when creating the filesystem on the USB disk using the -N option to mke2fs - a bit desperate

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@penguin42 I'm assuming this has to be done at formatting time, not on a disk that already has data.

CC: @dougmerritt

penguin42,
@penguin42@mastodon.org.uk avatar

@ColinTheMathmo @dougmerritt As far as I can tell, unfortunately yes.

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@penguin42 OK, I have a plan.

Thank you.

CC: @dougmerritt

penguin42,
@penguin42@mastodon.org.uk avatar

@ColinTheMathmo @dougmerritt Is it a cunning plan?

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@penguin42 Not Baldrick level, certainly.

Delete some old backups to let new backups start running again.

Once new backups have been taken, format another disk with more inodes.

Copy data.

Set up new disk for backups.

Reformat old disk with more inodes.

Now I have the required two backup disks to rotate.

Tedious, but doable in (mostly) background mode.

CC: @dougmerritt

dougmerritt,
@dougmerritt@mathstodon.xyz avatar

@ColinTheMathmo @penguin42
See mkfs.ext4 / mke2fs -N inode-count to increase the number of inodes.

Edit: -N as penguin42 said, not -I as I originally posted. Oops.

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@dougmerritt Can I do that on a disk that's currently got data?

I assume I'd have to unmount it, then do clever command line things.

CC: @penguin42

wrog,
@wrog@mastodon.murkworks.net avatar

@ColinTheMathmo

(apologies in advance for stupid-question):

Do we know the cd command didn't silently fail?
(i.e., are you in the directory you think you are? There might also be some special treatment for // that differs between commands. These are not URLs.)

(admittedly it's been a couple decades since I last had that kind of fail -- these are more NFS-ish kinds of problems)

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@wrog I am in the directory, the problem is I've run out of inodes.

wrog,
@wrog@mastodon.murkworks.net avatar

@ColinTheMathmo

so is this a case of

(0) we can rebuild the inode table without having to mess with the rest of the disk
(1) we need to reformat the disk (copy everything out and back)
(2) we need to recompile the kernel to allow bigger inode values
(3) we're fucked because inode_t is int32 and we have more than 4 billion files in the same filesystem

?

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@wrog Option 2.

We can format the disk with a larger number of inodes, but it's a reformat. Currently the number of available inodes is 152616960, and my disk is 40% full. So if I reformat with 3 times as many inodes, rounded up to 500M, then I should be OK.

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@wrog Thought that doesn't work, because with each new backup I use as many inodes as I have files, but don't use anything like that space, because anything that hasn't changed is hard-lunk to the previous backup.

I need to work on this.

But for now, I'll delete some of the old backups that I really don't need.

kludgekml,
@kludgekml@sunbeam.city avatar

@ColinTheMathmo could it be one of those dubious USB devices that reports itself as far bigger than it actually is? (Example: A friend once gave me a 1 Tb USB key she'd bought in Shanghai. It did not really have 1 Tb of storage)

ColinTheMathmo,
@ColinTheMathmo@mathstodon.xyz avatar

@kludgekml No, I've got the space, but run out of inodes.

https://mathstodon.xyz/@ColinTheMathmo/112226217179976669

Good shout, though.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • random
  • khanakhh
  • DreamBathrooms
  • thenastyranch
  • magazineikmin
  • osvaldo12
  • ethstaker
  • Youngstown
  • mdbf
  • slotface
  • rosin
  • everett
  • ngwrru68w68
  • kavyap
  • InstantRegret
  • megavids
  • GTA5RPClips
  • Durango
  • normalnudes
  • cubers
  • tacticalgear
  • cisconetworking
  • tester
  • modclub
  • provamag3
  • anitta
  • Leos
  • JUstTest
  • lostlight
  • All magazines