For anyone else who has the ASSEMbler Games archive, I thought I’d just leave up some notes on what I did and what issues I had while putting it all together.

I’ve since taken down my mirror as I didn’t have time to get it, let alone my GeoCities rebuild, working as well as I’d hoped. Maybe in the future it will reappear, who knows! Thanks to the good folk at Obscure Gamers for tracking this as it all went down. The YouTube archive credited to ASSEMBlergamers describes videos saved in 2019, before the channel was reported missing in September 2021 - lucky!

One of the biggest issues I hit on macOS was pathname length. I described it at the time as a “1024-character filename limit”, but that was the wrong term. An individual filename component can be up to 255 characters, while the traditional macOS pathname limit is less than 1024 characters. These recursive archive paths were blowing out the full path, not creating a single 1024-character filename. I also had names that did not display cleanly in the terminal, as the question marks above show. I described those as “non-printable characters” in my original notes; that was a separate cleanup headache.

Another example above. For the awkward names, I used the inode number to find the file without having to type its name.

The commands below are my historical cleanup notes. They include file deletion, so they are not paste-ready instructions for an archive you have not backed up and inspected.

We can identify the inode/file that needs to be removed with this command: ls -li

This finds files with inode number 99807 under the current directory and asks before removing each one. It removes the file, not the inode as a separate object: sudo find . -inum 99807 -exec rm -i {} \;

The next issue I came across involved links not working, possibly due to redirects that once existed.

In the middle of the page there is a link. I can’t remember what was broken and I have insufficient notes and no access to the data… so we’ll ummm skip over that one for the time being.

Looking at the above image, you can see that the index.html file isn’t found. That broken link blocks access to the attachment from this page and causes a little chaos when browsing threads, even though the attachment itself survives, as you can see below. I had this listed as “File listing not showing”. An annoyance certainly, but having it fixed was and will remain on my to do list indefinitely, unfortunately. Here’s the attachment for reference below.

Sitemap generation was a bit of a hassle. I had been using this command in Ubuntu 20.04: sitemap-generator 192.168.1.254:8081 -v. I vaguely remember doing a find and replace on the local IP address and port to match where the archive was hosted, changing 192.168.1.254:8081 to https://assembler.mcretro.net. Find and replace was my friend. Oh, and I think I kept the sitemap files to around 30,000 entries.

Another thing I had to do was modify the Apache configuration for assemblergames.mcretro.net to include the new port. My note for the virtual-host setting was VirtualHost *:80 *:8081, which is shorthand rather than a complete Apache configuration section.

And then we needed to edit the ports file: sudo nano /etc/apache2/ports.conf. To make the server listen on port 8081 at that local address, my note was Listen 192.168.1.254:8081.

It all became a bit chaotic when I started using a proxy server and I tried to merge it into my main server with deleterious effects…

Right?

I had been using the following to generate a sitemap, but it tended to fail part way through with an error. That’s why I switched over to the local IP method: sitemap-generator https://assembler.mcretro.net --verbose --max-depth 4

For reference here is the error I was getting:

/usr/local/lib/node_modules/sitemap-generator-cli/node_modules/sitemap-generator/src/index.js:88
      throw new Error(`Site "${parsedUrl.href}" could not be found.`);
      ^

Error: Site "http://assembler.mcretro.net" could not be found.
    at Crawler.SitemapGenerator.crawler.on (/usr/local/lib/node_modules/sitemap-generator-cli/node_modules/sitemap-generator/src/index.js:88:13)
    at Crawler.emit (events.js:198:13)
    at /usr/local/lib/node_modules/sitemap-generator-cli/node_modules/simplecrawler/lib/crawler.js:1264:25
    at FetchQueue.update (/usr/local/lib/node_modules/sitemap-generator-cli/node_modules/simplecrawler/lib/queue.js:229:9)
    at ClientRequest. (/usr/local/lib/node_modules/sitemap-generator-cli/node_modules/simplecrawler/lib/crawler.js:1247:27)
    at ClientRequest.emit (events.js:198:13)
    at Socket.socketErrorListener (_http_client.js:401:9)
    at Socket.emit (events.js:198:13)
    at emitErrorNT (internal/streams/destroy.js:91:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)

Note for future self - I really need to fix my code and pre tags to look better. Another failure looked like this:

/usr/local/lib/node_modules/sitemap-generator-cli/node_modules/sitemap-generator/src/index.js:88
      throw new Error(`Site "${parsedUrl.href}" could not be found.`);
      ^

Error: Site "https://assembler.mcretro.net" could not be found.
    at Crawler. (/usr/local/lib/node_modules/sitemap-generator-cli/node_modules/sitemap-generator/src/index.js:88:13)
    at Crawler.emit (node:events:390:28)
    at /usr/local/lib/node_modules/sitemap-generator-cli/node_modules/simplecrawler/lib/crawler.js:1264:25
    at FetchQueue.update (/usr/local/lib/node_modules/sitemap-generator-cli/node_modules/simplecrawler/lib/queue.js:229:9)
    at ClientRequest. (/usr/local/lib/node_modules/sitemap-generator-cli/node_modules/simplecrawler/lib/crawler.js:1247:27)
    at ClientRequest.emit (node:events:390:28)
    at Socket.socketErrorListener (node:_http_client:447:9)
    at Socket.emit (node:events:390:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)

Maybe it just wasn’t to be. And that’s alright. Onto the root directory structure, or at least as far as I could tell. There was a lot of cruft in there.

These are all that need to be indexed. Get directory listings from terminal on webserver and copy out the structure. All index.html files in these folders will need to be modified to remove sonicdude10 personalisation.

attachments	CONFIRMED - index.html prevents long file list load
categories	CONFIRMED
data		CONFIRMED
forums		CONFIRMED
help		CONFIRMED
js		CONFIRMED - Seems to be current / legacy javascript
members		CONFIRMED - Has member list
misc		CONFIRMED - Contains session-relative static files?
search		CONFIRMED - I added this, needs to be updated
styles		CONFIRMED - Xenforo stuff in here
tags		CONFIRMED
threads		CONFIRMED

As the forums were upgraded over the years we saw a shift in the way the URLs were formatted. This needs to be taken into consideration too.

2014 http://www.assemblergames.com/forums/showthread.php?50468-saturn-Model-search-help

2016 http://assemblergames.com/l/threads/saturn-model-search-help.50468/

2017+ http://assemblergames.com/threads/saturn-model-search-help.50468/

Something simple I had on my cleanup list was to track down 0-byte files and delete them, though I never did get around to it. The root folder is quite chaotic owing to all the other sites that were ripped along with ASSEMbler Games. This can be good in some ways, as images of hardware in threads were captured. I’m not sure how well implemented this was though.

Another curiosity I never worked out was this one. On the following link: https://assembler.mcretro.net/members/sonicdude10.65150/index.html you were able to see a link that references either the /l/threads or /threads.

Links might need to be rewritten between https://assemblergames.com/l/threads/ and https://assemblergames.com/threads/.

This could be fixed with some sort of redirect plugin, i.e.
try / 		THEN try 		/l/ 	THEN fail with 404
try /l/		THEN try 		/	THEN fail with 404

I never got around to looking into it too far though. It’s been good. Assembler Games was where I tended to hang around when I had free time during the mid 2010s. They ignited my love of retrogaming once more and gave me reason. They will be missed. All the best Kevin! :-)

Sources