A screenshot of Paper Mario 2

Here, we see trees, bushes, Mario, his friend, and a monster, all pretty clearly.  Now, let’s look closer at Mario’s head:

Zooming in on Mario's head

Looking closer, we clearly see the grid.  Each of these squares shows the average of all the colors that are underneath it in the “actual” picture being rendered.

Why do we represent pictures like this?  For one, it’s mathematically easy to deal with pictures like this when programming.  In most of today’s computing, we only represent colors in images with a set of three numbers between 0 and 255, one for red, one for green, and one for blue.  So (255, 0, 0) would be red, (0, 0, 0) would be black, (255, 128, 128) would be pink, and so on.  I pose a similar question for representing colors: why do we limit the resolution of each frequency to 256 digits?

The answer to both of these questions, of course, is that “the human eye isn’t bothered by it.”  We can’t see much of a difference between the shades of blue that would be represented by (0, 0, 190) and (0, 0, 190.5), so we don’t bother with it.  Note: This isn’t entirely true; in some cases, like in the 3D programming interface “OpenGL,” we represent the strength of the red, green, and blue channels by decimal numbers between 0.0 and 1.0.  So (0.743, 0.16513, 0.1665) would be valid for a “faded red” sort of color.  We still don’t have infinite precision with this method, but, like I’ve said, we don’t really need it.

Similarly, we’re usually okay with seeing a grid of square pixels to represent pictures, text, and so on, as long as the pixels are really tiny.  Even a modern “high definition” television, which is sold to make everything look realistic, uses a grid 1,920 pixels wide and 1,080 pixels tall to represent its picture–note that this is certainly an improvement from the old “standard definition” of 640 pixels wide and 480 pixels tall, but you might not have noticed the blockiness in it because most standard definition televisions blurred the lines between pixels by nature (there’s also interleaving, but I won’t get into that here).

But is a big matrix of 3-tuples of bytes the most efficient way to represent an image, never mind being easy to work with?  The example picture I gave above is 800 pixels wide and 600 pixels tall.  Each color takes three bytes to store its (red, green, blue) values.  So 800×600 pixels × 3 bytes/pixel = 1,440,000 bytes.  That’s one and a half million integers, each between 0 and 255, to give us a decent-sized picture.  Certainly, a picture being “worth a thousand words” holds in cases like these.

Thankfully, data compression can help.  Compression is a way to “summarize” a file, cutting out redundancies wherever practical, to achieve a smaller file that still sufficiently represents the original file.  An image compression algorithm might say, “there’s a rectangle at coordinates (63, 75) that is 104 pixels wide and 52 pixels tall filled with the color (37, 160, 67).”  Roughly, that’s two bytes to store the position, two bytes to store the dimensions, and three bytes to store the color–let’s say it takes one byte to store the instruction to draw a rectangle, and we are using 1+2+2+3 = 8 bytes to store a piece of an image that would’ve normally taken 104×52×3 = 16,224 bytes.

If the image uses 256 or fewer colors, we can compress even better, using a “palette” of sorts.  Basically, all the different colors that are used in the image are stored in the palette, which will be 256×3 bytes long, at most.  Then, in storing the image, instead of storing the color over and over again for each pixel, using three bytes apiece, we can simply use one byte to refer to which color in the palette.  So instead of saying (26, 45, 23), we can say “the 14th color in the palette” and, in some cases, save some space.

We can go even further on that idea.  Say the image uses more than 256 colors.  We can “merge” many similar colors to an average color in the palette, losing some information about the picture, but letting it fit the “256 color palette” paradigm from the paragraph above.  Nothing is particularly special about the number 256, mind you, other than that it’s the size of the palettes in the popular image compression formats GIF and PNG-8 (and that it’s the upper bound of numbers that fit in one byte).  16-color palettes are commonly used in video games on consoles like the SNES, Sega Genesis, and Nintendo DS.

Another interesting way to save space is to only save half the colors in the picture.  Perhaps surprisingly, you can still keep the compressed image looking reasonably close to the original in this way.  Using a method called S3TC (sometimes called “DXTn”), only two colors out of every 2×2 pixel block are stored, and the remaining two colors are “guessed” based on an average of the two that were stored. This is a very popular way to compress textures used in modern video games; just about every modern graphics processor has support for S3TC built into its hardware. In fact, Paper Mario 2 uses this format to store most of its textures:

A Mario texture extracted from Paper Mario 2, demonstrating S3TC

There are even more things we can do to compress a picture.  The JPEG format, developed by the Joint Photographic Experts Group, uses some advanced tricks that work well on photograph-like images to save space.  It often introduces artefacts, which are most noticeable when more aggressive settings are used to compress.  Often enough, though, these aren’t too noticeable on photographs and photograph-like images.  JPEG is horribly distorting for non-photographic images, especially those containing sharp transitions between colors, like text.

All of these schemes are retrofitted over the existing “grid” I described earlier.  The “lossy” schemes (shrinking the palette, guessing pixels, and using JPEG-like tricks) lose even more information on top of the existing loss of any information that didn’t fit into w×h discrete squares of ℤ₂₅₆ triplets. Only images that are intended to be made of such squares to begin with can be represented “losslessly” in such a grid. Note that these do exist, many artists produce what is called “pixel art” designed exactly for this medium.

However, in some cases, we can represent a picture even more accurately than raster graphics, often using even fewer bytes to do so.  This is possible when the picture is drawn as a “vector” image.  A vector art program stores the geometry of what is being drawn mathematically, so that curves, gradients, and so on scale nicely to any size when rendered.  Vector formats store exactly what was drawn, describing the curves, edges, and colors exactly as the artist intended, rather than storing an approximation of it in pixels.

Such an approach is only possible, of course, for graphics drawn on a computer.  Digital scanners and cameras with limited DPI and resolution can only store a certain number of “dots.” Perhaps in the future, a more advanced sort of camera could store a lower level of information, describing how the light passed through its lens differently, in such a way that the picture could be reproduced in a process similar to rasterizing a vector image. This is probably rather far-fetched, though; I’m not exactly a photography expert.

TODO: Discuss similarities to audio and music.

Several emulated video games running on the sides of a Compiz desktop cube

Hooray for the PC. Nobody would actually play like this, mind you, but the fact that they all run without conflict is certainly impressive-looking!
I’d say “see if you can name all the games!” but many of them are showing title screens. See if you can name all the consoles being emulated!

The Game Boy Advance is generally used for this, presumably the last of the “Game Boy” series of hardware from Nintendo, because it has hardware backwards-compatibility to all the software of the original Game Boy, and because the older Game Boys are just that–older, harder to come by, and more likely to be broken.

The Nintendo DS isn’t backwards-compatible with the older Game Boy software, but it is backwards-compatible with GBA software. Since the GBA exposed the original Game Boy’s PSG to GBA-specific software, the DS must also provide that functionality, so the same type of PSG was included in the DS hardware. This is fortunate, since the DS is relatively easy to develop for, thanks to devkitPro. An example of the use of the PSG in the DS can be found in the very nifty AXE, written by PinEight.

I have no experience with other existing 8-bit music composition software, but I do have knowledge of music theory (thanks to my brilliant professor Andy Brick, who coincidentally has composed and conducted quite a bit of video game music–check it out!), and I am trying to work both of these facts to my advantage in an effort to make a unique and useful interface for my application. One of my ideas for a user interface involves a selection of chords that would be valid in syntactic progression, where the program takes care of all the voice leading rules after the user has given all the specifics of what he or she wants to be used.

There are some other ideas floating around in my head, too. For now, the interface is very simple: a proof of concept using Zelda 64 “Ocarina-like” controls (the C-buttons map to A, B, X, and Y, and the A button maps to the DS’ R). I’ll post more on this project as I make progress on it. In the meantime, check out some of Bit Shifter’s Information Chase album, available for download for free! (In MP3 format, I know, I’m sorry)

In a normal semester, there are many events run by student organizations, clubs, and the like, where each use part of their budget to get some pizzas to attract more people to their events. This is the most cost-efficient way to obtain pizza (short of someone finding some way to get paid to eat pizza (I’m all ears–if you’ve heard of anything like this, please comment)). It’s also good because the pizzas often come from one of the many fine pizza restaurants in Hoboken.

The second-most efficient method relates to the dreaded meal plan. Any student here living in on-campus housing that doesn’t have a kitchen in his or her room, with certain exceptions, has to be on the meal plan. The meal plan has you pay a constant price up front for an “unlimited” number of times to be let in to the dining hall. The dining hall has an all-you-can-eat buffet, with pizza and various other foods (that rotate day to day). The pizza is okay, I certainly don’t expect to hold it up to proper Hoboken Italian restaurant standards; that would be rather ridiculous to expect of a campus dining hall. But if it’s already been paid for, cashing in on the meal plan is a viable method by which to acquire pizza.

Unfortunately, there’s not always a free-pizza event going on, especially during the summer. Worse yet, the dining hall is closed during the summer. This means the pizza money has to come out of my own pocket. There are many places to choose from here, and I need to determine how I can get the most pizza for the least amount of money. So, I wrote a small, trivial application in C++ (which should be understandable to beginners) that compares an arbitrary number of pizza places based on the amount of pizza you get for the price you pay (per slice).

#include <algorithm> // for sort()
#include <iostream> // for simple i/o
#include <cstdlib> // this is really only here for aesthetic purposes
#include <vector> // nicer to use than arrays, especially with stl algorithm
#include <cmath> // for M_PI

struct pizza_attribs {
	int num;  // index so we know who's the best after sorting
	int slices;  // the number of pieces, not the number of cuts.
	float diameter; // width of the pie
	float slice_cost;  // slice price
	float bang_for_buck;  // = (slice area)/price
};

using namespace std;

bool pa_compare(pizza_attribs a, pizza_attribs b) {
	return a.bang_for_buck > b.bang_for_buck;
}

int main() {
	vector<pizza_attribs> pa_list;
	int count;

	cout << "Be consistent with your units!" << endl << endl;
	cout << "How many places to compare? ";
	cin >> count;
	for(int i = 0; i < count; ++i) {
		pizza_attribs pie;

		pie.num = i;
		cout << "[pizza place #"<<i<<"] Price of a slice of pizza? ";
		do { cin >> pie.slice_cost; } while(pie.slice_cost <= 0);
		cout << "[pizza place #"<<i<<"] Number of slices in a pie? ";
		do { cin >> pie.slices; } while(pie.slices <= 0);
		cout << "[pizza place #"<<i<<"] Diameter of the pie? ";
		do { cin >> pie.diameter; } while(pie.diameter <= 0);

		pie.bang_for_buck = ((M_PI*pow(pie.diameter/2, 2))
		                     / pie.slices) / pie.slice_cost;

		pa_list.push_back(pie);
		cout << endl;
	}

	sort(pa_list.begin(), pa_list.end(), pa_compare);

	cout << "The results are in!" << endl;
	for(int i = 0; i < pa_list.size(); ++i) {
		cout << "[pizza place #"<<pa_list[i].num<<"] bang for buck: "
		  << pa_list[i].bang_for_buck/pa_list[0].bang_for_buck << endl;
	}

	return 0;
}

For example, the two closest pizza places to me right now are Benny Tudino’s and H&S Giovanni’s. Benny’s slices are taken from a 28-inch pie, sliced into 8 pieces, and cost $3.00 apiece. Gio’s slices are taken from a 22-inch pie, sliced into 8 pieces and cost $3.00 apiece (one can already intuitively figure out that you’d get more pizza from Benny’s). According to this program, you get an estimated 61.7% of the pizza at Gio’s that you would at Benny’s.

Now, this program makes some unreasonable assumptions: Namely, it assumes that all pizzas are of the same thickness, and it doesn’t factor in how good the pizza tastes. However, it makes objectively comparing pizza slices by area versus price a little bit easier.

TODO: Factor in distance to restaurant?

Some time ago, navaburo and I were hanging out in my dorm room, and we ended up thinking about the concept of writing an AI to find the best (shortest) path through a video game. A couple days later, we went down to the Linux Lab and fleshed out the idea on paper.

For those unfamiliar with the concept of “Tool-Assisted Speed-running,” it is a process done by some video game enthusiasts in which they play a game on a an emulated (or “virtual”) machine, using special features of the emulator to let them more easily make an almost “perfect run” of the game, where they complete the primary objective as quickly as possible. This usually involves slowing down the game to give them the opportunity to take shortcuts and pull off maneuvers that wouldn’t have been possible with the limitations of human reaction time and reflexes. Sometimes, they also use “luck manipulation,” which involves taking a snapshot of the machine’s state (also known as a “savestate”) before an event that relies on a “random” number, and restoring the machine to that state again and again, until a desired number is generated. Similarly, they often use this functionality to rewind the gameplay in case they make a mistake. When a speed-runner completes a game this way, he or she might post a replay of the “run” on the internet to share with others; there are communities on the web in which speed-runners will compete to complete classic video games with the fastest times they can manage.

navaburo and I theorized that, using an emulator for a given video game console, we could calculate the fastest possible path through the game by using a search on all of the possible inputs starting from the machine’s initial state. This is most likely very impractical to compute, even for the relatively sophisticated computer hardware we have today. Nevertheless, we decided that as long as we could provably calculate the optimal path to do some trivial task, we could also calculate the optimal path to the very end of the game. For proof of concept, we decided on Super Mario Bros. for the NES, and we’d call the program “OptiMario.” We spent some time looking into ways to bootstrap our search into the input functions of various open-source NES emulators, including the popular FCE Ultra and Nestopia, before deciding that for our purposes, the simpler (and lesser-known) NEStra would suffice. So far, we’ve hacked the beginnings of a foundation for this project into NEStra.

I won’t go too much into our methods just yet, mostly because we still haven’t implemented it and the exact method we end up using might change. However, in theory, it seems we might be able to create an artificial “ultimate speed-runner” that is able to complete a game in the fastest possible time, perhaps even using glitches and shortcuts previously unknown (if it can be done with a valid input string, and it shaves seconds off the final elapsed time, such a program (properly implemented) will find it).

So good of you to join me. I’ll be posting here about my various projects and whatever else I think is interesting enough to post.