I had a manager once tell me during a casual conversation with complete sincerity that one day with advancements in compression algorithms we could get any file down to a single bit. I really didn’t know what to say to that level of absurdity. I just nodded.
That’s precisely when you bet on it.
It’s like that chip tune webpage where the entire track is encoded in the url.
Link?
Are you trying to get rickrolled?
If you have a tub full of water and a take a sip, you still have a tub full of water. Therefore only drink in small sips and you will have infinite water.
Water shortage is a scam.
There is a water shortage?
Exactly!
Out of context, but this video showing the amount of freshwater on the planet in perspective was eye opening for me… I see water availability different since.
Don’t worry, global warming is desalinating the water so it will all be fresh in time 🙏
Stupid BUT: making the font in LibreOffice bigger saves space. so having 11 is readible but by changing the font size to like 500 it can save some mb per page
I dont know how it works, i just noticed it at some pointEdit: i think it was kb, not mb
Have a macro that decreases all font size on opening and then increases all again before closing.
Follow me irl for more compression techniques.
per page
I mean, yes. obviously.
If you had 1000 bytes of text on 1 page before, you now have 1byte per page on 1000 pages afterwards
You could always diff the XML before and after to see what’s causing it.
Reality is stranger than fiction:
I was sort of on Mike Goldman (the challenge giver)'s side until I saw the great point made at the end that the entire challenge was akin to a bar room bet; Goldman had always set it up as a kind of scam from the start and was clearly more than happy to take $100 from anyone who fell for it, and so should have taken responsibility when someone managed to meet the wording of his challenge.
Nice stuff.
I got sold on the :
EOF
does not consume less space than “5”because, even though the space taken by the filesystem is the fault of the filesystem, one needs to consider the minimum information requirements of stating starts and ends of files, specially when stuff is split into multiple files.
I would have actually considered the file size information as part of the file size instead (for both the input and the output) because, for a binary file, which can include a string of bits which might match an
EOF
, causing a falsely ended file, would be a problem. And as such, the contestant didn’t go checking forcharacter == EOF
, but used the function that truly tells whether the end of file is reached, which would, then be using the file system’s file size information.Since the input file was a 3145728 bytes and the output files would have been smaller than that, I would go with 22 bits to store the file size information. This would be in favour of the contestant as:
- That would be the minimum (hyh) number of bits required to store the file size, making it as easy as possible for the contestant to make more files
- You could actually go with 2 bits, if you predefine MiB to be the unit, but that would make it harder for the contestant, because they will be unable to present file sizes less than 1 MiB, and would have to increase the file size information bits
On the other hand, had the contestant decided to break the file between bits (instead at byte ends), instead of bytes (which, from the code, I think they didn’t) the file size information would require an additional 3 bits.
Now, using this logic, if I check the result:
From the result claimed by the contestant, there were 44 extra bytes (352 bits) remaining.
+ 22 bits for the input file size information - 22*219 bits for the output file size information because 219 files
so the contestant succeeds by
352 + 22 − (22 × 219) = −4444
bits. In other words, fails by 4444 bits.Now of course, the output file size information might be representable in a smaller number of bits, but to calculate that, I would require downloading the file (which I am not in the mood for.
And in that case, you would require additional information to tell the file size bits. So;- 5 bits for the number
22
in the input - 5 bits for the size of the file size information (I am feeling this won’t give significant gains) and rest of the bits as stated in the first 5 bits, as the file size bits
- you waste bits for every file size requiring more than 16 bits to store the file size information
- it is possible to get a net gain with this, as
qalc
says,log(3145728 / 219, 2) = (ln(1048576) − ln(73)) / ln(2) ≈ 13.81017544
But even then, you have
352 + 5 + 22 − (5 + (14 × 219)) = −2692
for the best case scenario in which all output file sizes manage to be under 14 bits of file size informations. More realistically, it would be something around352 + 5 + 22 − ((5 + 14) × 219) = −3782
because you will the the 5 bits for every file, separately, with the14
in this case, be a changing value for every file, giving a possibly smaller number.
If instead going with the naive 8 bit
EOF
that the offerer desired, well, going with 2 consecutive characters instead of a single one, seems doable. As long as you are able to find enough of said 2 characters.
After going on a little google search, I seem to think that in a 3MiB file, there would be either 47 or 383 (depending upon which of my formulae was correct) possible occurrences of the same 2 character combination. Well, you’d need to find the correct combination.But of course, that’s not exactly compression for a binary file, as I said before, as an
EOF
is not good enough.This was too damn funny for what I expected it to be
That story is immediately what came to mind.
It’s all fun and games until your computer turns into a black hole because there is too much information in too little of a volume.
Even better! According to no hiding theorem, you can’t destroy information. With black holes you maybe possibly could be able to recover the data as it leaks through the Hawking radiation.
Perfect for long term storageCan’t wait to hear news about a major site leaking user passwords through hawking radiation.
i love this comment
Really-long term storage :)
Longer than your lifespan, too.
Longer than the life span of the most long-lived star. By orders of magnitude.
Awesome idea. In base 64 to deal with all the funky characters.
It will be really nice to browse this filesystem…
The design is very human
Or use yEnc: https://en.m.wikipedia.org/wiki/YEnc
Broke: file names have a max character length.
Woke: split b64-encoded data into numbered parts and add .part-1…n suffix to each file name.
Browse your own machine as if it’s under alt.film.binaries but more so
I’d go with a prefix, so it’s ls-friendly.
each file is minimum 4kb
(base64.length/max_character) * min_filesize < actual_file_size
For this to pay off
Reminds me of a project i stumbled upon the other day using various services like Google drive, Dropbox, cloudflare, discord for simultaneous remote storage. The goal was to use whatever service that has data to upload to, to store content there as a Filesystem.
I only remember discord being one of the weird ones where they would use base512 (or higher, I couldn’t find the library) to encode the data. The thing with discord, is that you’re limited by characters, and so the best way to store data in a compact way is to take advantage of whatever characters that are supported
GmailFS was a thing
Store the data in pings that constantly get resent to keep the data in the internet
I remember a project where someone booted Linux off of Google Drive. Cursed on many levels.
What about a hard drive made of network pings?
this is actually a joke compression algorithm that compresses your data by one byte by appending it to the filename. (and you can execute it as many time as you want)
Too bad I can’t remember the name.
Obligatory “pi hasn’t been proved to be normal”
Oh, I guarantee that pi is 100% normal. Just not necessarily in the base you want it to be normal in.
I don’t know of a proof that pi is normal in any base (even non-integer bases) so I’d be interested to see on what basis you can guarantee it.
It’s somewhere in pi. Wait a moment while I look it up.
Relatable tbh
Pi being irrational and not normal makes it the most relatable number
we could also use it as a file system
deleted by creator