A preview my wonderful Patreon supporters got back in July:
This week I have been mostly working on the Penultimate Cartridge.
Every year or so, we decide it is time to refresh the games list, this time we have a few new games to add, including the great Thrust-like but utterly unpronounceable Gravzronox from Misfit (which TFW8b refers to as "Gaviscon" as he has given up trying to pronounce it).
Also from Misfit, the upcoming game which is not called "Doctor Michael Portillo".
There are also hints that we might have something new from Hewco, and one or two from me, including the VIC20 version of Perilous Swamp, and something else I started ages ago that I must get around to finishing.
Edit: this was referring to my rewrite of TimeBomb in assembler.
Making Space
The problem is, the Penultimate Cartridge is full. We have two megabytes of EPROM space, providing about 300 titles at the moment (I haven't counted them in a while, but that works out an average of about 7K each, so it sounds about right).
Edit: the previous release has a total of 241 titles in the A-Z list, not including adventure games, board games, paddle games, utilities and programming tools, all of which are on separate menus.
It usually comes down to what games can be removed to make way for the new ones. Sometimes we find one that have managed to avoid being culled previously as they are still quite good, and ones that probably should go, but we have to keep because of historical significance etc.
For example, in the last release, we finally got rid of the "official" version of Q*Bert, because it really isn't very good.
No effort is made to deal with colour clash, simply adding a black square into the yellow diamonds when you hop onto them. It was also NTSC only with no adjustment for PAL systems.
We tried to keep that because it was Q*Bert.
In the end, we have Topper, which is a much better version.
Even Max is better.
Both better versions of Q*Bert than Q*Bert, so it went, and that was only an 8K ROM.
The same sort of rules apply, so we have been going through trying to find space for two 32K ROM games, a couple of 8K games and potentially more.
We almost got to the point of considering removing some of the less visible things.
Things like the text adventures and the paddle games are both on separate menus rather than the main list.
One target was Pinball Spectacular.
Not a bad pinball simulator, but it was two 16K ROMs, one for PAL and one for NTSC, which seemed to be very different internally, with not much direct commonality.
Other games where there were PAL and NTSC versions turned out to be a handful of bytes different, and I was able to store the NTSC version and patch those few bytes to create a PAL version from the menu program when it was launched.
So Pinball Spectacular in total was 32K. It that was to be removed, that would get us one of the new games, what about the rest?
An Idea
Then I had an idea.
I had previously been able to make some space by using a program called Exomiser to compress some of the programs, but it was only able to deal with PRG files from 8-24K, and we found that it didn't work on some of the programs we tried, and others were barely reduced in size, but where it did work, it worked well, some games went down to less than half their original size
I was looking through at some of the older cartridge images in there, ones from the 1980s, like Avenger and Donkey Kong etc.
Some of those looked to have space in them, or runs of $00 or $AA of $FF etc.
I wondered if I could use a simple compression algorithm like Run Length Encoding (RLE) to compress these files.
This is an old technique where you look runs of repeated characters, so a run of twenty $FF characters, could be replaced with a single $FF and a count of 20.
That works really well with data which is only long runs of the same character, large monochrome bitmaps for example.
The worst case is all the characters are different, so you get 1x $17, 1x $0F, 1x $12, 1x $0C, 1x $40 etc. Best case is they are all the same character, so you get 256x $42.
With a 256 byte data block, the best case is 2 bytes, a massive saving, but the worst case if 512 bytes, double the original size.
I tried this out and it was pretty awful, most of the data blocks were over 500 characters.
Even in the cases where it was sort of effective, it only barley did better than a straight copy.
A few cases were a bit shorter, but certainly no two-byte results.
Another Idea
Then I had another idea.
Why don't I process the 256 bytes blocks, and if the compressed length is >= 256 bytes, then I set that block as a straight block copy. If it is < 256 bytes then I store the RLE version.
On decompression, I will have a table of each block type, start address and destination.
It will either by a straight block copy or an RLE decompression.
I tried it.
It was rubbish.
All but two of the blocks were straight copies, and with the overhead of the block table, the file was larger than before.
A Third Idea
The third idea burned down, fell over, and then sank into the swamp.
The Forth Idea
But the forth idea, the forth idea worked.
It is a little convoluted, but basically I need to have an 8K block of ROM at address $A000 to run the game.
I can't Exomise that as it's not a PRG file and doesn't have the normal load addresses.
But, what if I create a program. A program which consists of a small copy routine, then an 8K block of data.
All* it will do when it runs, is copy the immediately following 8K block of code up to $A000 in RAM, then poke at the Penultimate cartridge to turn that 8K block read-only and reset the VIC. When the VIC resets, it sees a cartridge ROM signature in the right place at $A000 and runs the game.
(* not quite all)
I put that together as a test and ran it, and it decompressed to 50% of the original size.
I tried it, and it ran, took a second or so to decompress, you could see the tell-tale flashing A in the bottom right hand corner. I had arranged it so that it would not clear the screen, so the LOADING message still started there.
And then the game started as if it was running from a cartridge.
Great, it turns out there are quite a lot of cartridge games that this could be applied to.
I tried a few more games and most of them were going down to about half.
This time it was right, it would work, and no one would have to get nailed to anything.
A Bonus
There is a little bonus with this.
I mentioned previously that I was patching some of the PAL/NTSC ROMs, well, now I have this little loader program running and copying the cartridge image into RAM.
That is an ideal place to put the patching code. I can do a simple test to read the VIC 20 ROM and check what version it is an then apply the patch if necessary, before making the RAM read-only. (RORAM?)
A quick refresher on the way the VIC20 deals with screen positioning:
It doesn't.
Well, rather it can, but it does not initialise anything when a cartridge is detected, so it is down to the game designer if they call the standard KERNAL routines and get the standard size screen, or if they do it themselves.
Glossing over lot of detail, the two relevant registers in the VIC chip, are $9000 and $9001. These contain a horizontal and vertical offset of the screen respectively. Basically, the width of the left border, and the height of the top border. How far the top left character in the text area of the screen will be from the top left corner of the actual screen.
On a normal VIC when it boots to the BASIC READY prompt, the values will be $0C and $26 for PAL.
And $05 and $19 for NTSC.
If you happen to put an NTSC KERNAL ROM into a PAL VIC, you would get the screen in the wrong place.
And vice versa.
There is a table of default values in both versions of the the KERAL ROM at $EDE4. And a routine at $E518 which will copy those into the VIC, setting up the normal screen.
Some games will call the routine to setup the VIC to the default values.
Others will read the table and use the values to determine if this is a PAL or NTSC system then apply their own values.
The reason many did not, or could not, use the default values, is the offsets change depending on how many rows and columns you have the screen set to.
The VIC is very configurable, so you see games going to all the extremes.
Some tall and thin, some short and wide.
Some max it out to get the most screen area, so much so they don't fit onto the smaller NTSC window.
Many just setup the VIC how they want and if you happen to be in the wrong region, it is going to be in the wrong position on the screen.
Some games, particularly NTSC cartridges, had the option to use arrow keys or sometimes the joystick (or in one case F5 and F7) to adjust the screen centering to suit your system.
All of this could have been avoided if they had a well documented and well publicised KERNAL function you could call that would setup the screen for a given row and column size and set the appropriate offsets for your region, either by calculation or lookup table.
Since I was looking at writing code to patch the games that already had PAL and NTSC versions, this was also an ideal place to patch games that did not have both PAL or NTSC versions.
There were many games, including things like Centipede, that did not have a PAL version, or any adjustment, so with a patch as part of the loader, I can bring many more games that were previously not available in the other region.
I could also pre-set the starting points for games that had the arrow key or joystick adjustments so they would start in an appropriate position for the current region.
User Experience
TFW8b was initially a little concerned that the decompression delay might affect the user experience, but with this change affecting more than half of the games I was looking it, it changes the user experience it was previously:
1) Select Game
2) Load Game
3) Adjust Screen
4) Play Game
It is now:
1) Select Game
2) Load Game
3) Game Decompresses and Patch if Necessary
4) Play Game
I think that is an acceptable swap. Ideally, the decompression step would be eliminated, but unless more storage magically appears, this seems the best option.
It also freed up an awful lot of space in the cartridge.
Enough for the two new 32K games, and about 8 more 32K games (or 32 more 8K games).
(N.B. we did find a number of nice recent games from one author that we wanted to add, but unfortunately we could not get in touch with them so they were removed from the cartridge at the last minute)
That should keep us going for this update and the next one or even two, before the Penultimate +4 eventually appears in a few years with some yet to be imagined vast storage data cube.
Update: Totaliser Results
The A-Z list does not include adventure games, board games, paddle games, utilities and programming tools, all of which are on separate menus.
The previous release has a total of 241 titles in the A-Z list. Removing NTSC-only, that is 224 on PAL systems, and without PAL-only, that is 190 on NTSC.
The updated release has a total of 245 titles in the A-Z list (243 on PAl systems, 215 on NTSC)
You can see TFW8b's new video showing the updated cartridge in action on the You Tubes (look out for the flashing letter A bottom right of the screen during loading).
Adverts
The VIC20 Penultimate Cartridge +3 DCR is available now from the Future Was 8 bit.
All Tynemouth products are still available, and I can ship worldwide.
Including the limited edition 10th anniversary recreation of the very first Penultimate Cartridge:
International shipping is a bit complicated at the moment, so if you want to order from outside the UK, please contact me using the contact link at the top of this page.
See here for more information:
Sorry to have to do this, and I know this is going to affect sales, but please understand it is the only option I have right now.
Patreon
If you enjoy posts like this, you can support me via Patreon, and get access to advance previews of blog posts, and exclusive posts.
You also get progress updates on new projects and other behind the scenes updates, as well as access to my Patreon only Discord server for even more regular updates, and to discuss your own projects.
