Cut the fat, NSData diet

Fat XML
I'm a long time user of the XML format for transferring data between client and host. It's human readable, can be edited by hand easily in most cases, and it conveys objects and nesting well. So in the past I was often designing a object scheme and custom parsing loops that solved the issues that faced constantly updating applications and site I've been in charge of through out my flash career. The problem I was always concerned with when designing the scheme is how much data was just being taken up by open tags, close tags, and parameter defines. This became exceptionally a problem when working for the company responsible for "MMO crack". I came on board to find that they where converting C objects out to XML. Now I wasn't programming in C yet myself and had lacking experience there, all my objects where far more dynamic in action script. But when I was tasked to parse open some game data I was shocked to see my work be brought to it's knees when getting the users character roster and sub inventory arrays that were compiled into XML Data and downloaded to the Flash client. This may seem to make sense you need to know this information in order to play that game. But for a new character that had no team members, and starter inventory, the XML data surely didn't need to be 2-3mb to tell me I'm a noob. It turned out that when the  object structures are being defined, the play character with some properties, has a max inventory lets say 30 items, and a roster of teammates with a max of say 30, each with their own properties and inventory of 30. So in memory this struct was created with these arrays of array defined. And even though the inventory had maybe two items, and the team roster empty, the space for this storage had been allocated. This left the company's choice XML writer with the task of creating a node tree for every allocation, even if the value was null. Not only was the XML nodes verbose, the property attributes fully spelled out, but every download came in like a template worksheet to be filled in and a later time.

Average JSON
I had always made it a conscious effort to slim down the data that was being sent back and forth in applications I designed, even though this was often not my choice when working at large companies. JSON came along in the midst of all this work and I found it a great way to slim down a lot of the parseable data. A few iOS projects that I started on in 08' I was doing a lot of tracking multiplayer turns moves on a PHP/SQL backend. I was comfortable using JSON and NSDictionaries to convert the data because I could understand how it was being handled on both places. But I did alway prefer to have my classes with their instance variables. I felt like KVO in NSDictionaries was just a long way to get at values in code and made for more lines. I also want to avoid waiting and loading screens, so I separated many requests into small object calls, and take it even further I got on a kick where I would make property names only one or two characters long. If the property name was 'int character_id = 500;' I would hold the value in a temporary NSDictionary to encode the property to {c:500}. Every little bit helps I figure.

Thin Binary Data
This year I have been getting more ambitious about using some Multiplayer GameCenter features, wether it be live games, or turn based games. Now the out of the box features of encoding NSDictionaries to a compatible NSData binary I think leave you somewhere in-between the first to parts of this post. Where you can define small objects maps, but this its coded into a XML like string before its coded to binary. Game data needs to be condense to make a quick round trip in peer to peer or to fit in a maximum size allowed for turn based storage. I've come full circle now to better understand C structs and have come to some pros and cons about using them to transmit game data. What really helped me prepare for the use of C structs as NSData was the GKTank sample. Now if your not entirely sure whats going on in this sample like I once was, then hopefully I can help you understand it.
First you must be comfortable using C structs to define your properties. If you are very used to the way NSObjects and Class in Obj-C act for you when you retain and release, you have to be aware of when you assign a struct you are making copies and may be dealing less in pointers. Lets talk about GKTank tankInfo:

typedef struct {

CGPoint tankPreviousPosition;

CGPoint tankPosition;

float tankRotation;

float tankDirection;

} tankInfo;

This struct has four properties, but two are also structs. CGPoint is a C struct that just contains two sub float values. And understanding how this all really boils down in memory helps understand how much data your saving or sending. A float is 4 bytes, this example of struct ultimately consists of 6 floats. 4*6 = 24 bytes to send this struct. Just to compare that point here, the resources out there tell you normally would want to run your NSDictionary through a NSKeyedArchiver which can write to a NSData. An example of this:

    NSDictionary* tank = [NSDictionary dictionaryWithObjectsAndKeys:

                         [NSNumber numberWithFloat:32.f], @"oldx",

                         [NSNumber numberWithFloat:32.f], @"oldy",

                         [NSNumber numberWithFloat:32.f], @"newx",

                         [NSNumber numberWithFloat:32.f], @"newy",

                         [NSNumber numberWithFloat:32.f], @"rot",

                         [NSNumber numberWithFloat:32.f], @"dir", nil];

    NSMutableData *data = [[NSMutableData alloc]init];

    NSKeyedArchiver *archiver = [[NSKeyedArchiver alloc]initForWritingWithMutableData:data];

    [archiver encodeObject:tank forKey:@"theTank"];

    [archiver finishEncoding];

Same six float values, for a total of 436 bytes to hold the values that map the key values. So how do we get the struct into a NSData object so the rest of the Objective-C framework can handle it. NSData has a handy way of making a copy of the binary data of the structure provided we can tell NSData how many bytes the object is.

tankInfo mytank = tankInfo();

    mytank.tankPreviousPosition = CGPointMake(32, 32);

    mytank.tankPosition = CGPointMake(32, 32);

    mytank.tankRotation = 32;

    mytank.tankDirection = 32;

    NSData *data = [NSData dataWithBytes:&mytank length:sizeof(tankInfo)];

[myGKSession sendData:data toPeers:peers];

This method Data with Bytes lets us send in the pointer argument of our struct which is giving it the first byte to start from. The length lets the NSData know how far in memory to copy, so we need to know how many bytes our struct is. We did the math earlier and could enter 24 for the length. But C offers a function sizeof() which looks up the definition table of the struct type and returns it for us. Whats great about a struct is when allocated it's properties are lined up in memory so that each of the 6 floats are stacked back to back. How it is aligned is not saved in the memory block with the 6 values but the structs definition itself acts as a parser/de-parser for the memory. Even if the floats are back to back, the definition is the road map to know that the first and second floats are part of the first property, the third and forth floats are the second property, etc. When you get this NSData on the other side it needs to be converted back into a structure so it's easy to access again.

- (void)receiveData:(NSData *)data fromPeer:(NSString *)peer ... {

    tankInfo recievedTank;

    if ([data length]) [data getBytes:&recievedTank length:sizeof(tankInfo)];

}

When we create the instance of tankInfo the memory for 24 bytes has been made available but like is random data or set to null. Like before this now copies the bytes into the pointer location which has already been allocated the line before and the data that just came in can now be accessed and changes by accessing the struct properties and all we sent over the wire was 24 bytes (give or take any gamecenter overhead).

With my mind well wrapped around this C struct concept I can wisely get a lot of game data send over the wire quickly, or stored on Apple servers when making a turn based game. Apple has set a turn based miscellaneous data limit to 4KB. Now for some these seem unreasonable, but those are often the ones I see trying to KeyArchive encode a big Dictionary and it makes sense that they are hitting the limit quick. I've headed into planning my game data storage and for the most part have not come close to needing to worry about how much binary data I need to get my games conveyed. up to 4 players, up to 70 turns, in under 880bytes.

The Con
I know I'm picking a threads here, but now I've come back to the issue I once though obsessive when working for the MMO. I'd love to hear input there from any others that deal in structure binary. I've designed my game data struct to handle these 4 players and the 70 turns possible to complete a game. To define this maximum possible amount of turns I an array of turn structs. This again cause the data to be allocated, and empty early on. An average game will be complete in 60 turns, and as the turns progress the bytes in the turns array are being written to something other Null, but in order to know the size of a matchData struct the turns need to be defined with 70 elements. Only fortunately is this set of struct small and the whole combo comes to 880bytes. But thats 880 on turn 1, 2 .. 54 etc. If later I take on any larger game storages I may come back to talk about how to sub parse the first few bytes in the NSData to help predefine the amount of elements that should be expected, making the struct more variable.

struct matchData {

    unsigned int seed;

    unsigned char playerCount;

    unsigned char turnIndex;

    playerData players[4];

    turnData turns[70];

    

    matchData() : turnIndex(0), playerCount(2) {}

};