About Where To Not Use Json

Posted by Vic's Blog on July 9, 2015

I believe that many of you have heard about the most recent drama to the Modding Community that came upon us in late 2014, in the name of JSON. I will try to sum up some of the major points, and try to explain my point of view as best as I can: Why I think that JSON isn’t the right thing to use here.

I realize that I’m a bit late to the party here, but I don’t think there are any extracts around that sum up on the situation, so I thought that I might as well try it with this one. Be warned before you are starting to read this. My language might be flippant, but this is the usual way I write, and I wouldn’t enjoy it if it wasn’t for that, so don’t feel offended. If you are a hardcode JSON-ist, I won’t be able to change your opinion anyway, so you might as well stop reading now.

My motivation to this are the recent disclosures on 1.9 due to Minecon 2015, mainly this tweet that I’ve found:

One could say “Well, this is obviously better than 80 fire models so it must be a good thing” now, but I’m sorry, this is not the case. It got worse in several aspects as they’ve switched from less-wrong and miserable to hellfire and less-miserable

But more of that later. First I’d like to cover what the initial problem was, because you know, never change a running system.1

1 : Unless your code is messy, fucked up, doesn’t work, is written in the wrong language (php, Lisp) or insults cute kittens. Mojang’s code probably did a few of those things at the same time.

What it is that doomed us in the first place

The user pretty much wasn’t… Unless you he had the urge of changing how the model of a furnace looks like, which is a neat thing to do, and usually a given in most other games out there just by the fact that they have a well designed engine. Note that this doesn’t mean that they allow the user to change any of the assets, but they could do so at any point in time, without hesitating.

And here we are at the core of the problem.

In <1.8 block rendering pretty much looked like this:

Tessellator global // I hold all the vertex data to be dumped to the GPU

for every chunk in all_loaded_chunks do
  for every block in chunk do
    switch block.renderid
      case 0 :
        // add data for a standard block to global
      case 3 :
        // add data for some fire to global
      case 24 :
        // add data for a cauldron to global
      ...
    end
  end
end

global.flush() // Send everything that the buffer holds to the screen

Not that simple of course, there is some logic going on that determinates if a chunk is visible, code to do lighting, etc… but you get the idea

There are multiple problems with that design. Well, it is simple to write at first, but a wise man once said that the simpler something is first go, the harder it gets to maintain it. You can’t just go ahead and replace your furnace model because well, there isn’t even a furnace model to begin with. In order to do a change that sounds as simple as that, you’d have to decompile the game, search through the render code and change it there. On top of that, most of Minecraft’s rendering is done using Divide and Conquer, which means that it is rather fast because it does a lot of branching and very little math2, but it’s also terrible to read through or even worse, to change afterwards.

This essentially breaks a rule called Seperation of Concerns which allows a company to exist in the first place, where a whole bunch of people can work on the same thing without having to get into each other’s way. It allows for people like artists to do their artist work without having to deal with a bunch of code, eating all of their time because well, Models are assets.

Another thing that is suboptimal here is that this code can’t be multithreaded. Multithreading essentially means to work in a team on something, you could visualize it as a bunch of minions that draw a painting together. It is faster than one minion drawing it but you have to coordinate them properly.

Both of these problems got addressed for 1.8, more or less complete3. Especially multithreading got a lot better. But then there was a terrible design choice that made a lot of us shiver and shake our heads: Using JSON for the newly incorporated model assets.

2 : I don’t know how much of this is influenced by the fact that I’m looking at decompiled code, so please correct me if I’m wrong on that one.
3 : I’m referring to the common theory that 1.8 was a rushed release that was pushed out during a refactor period for pleasing the gadflies.

Why JSON isn’t the binary file killer

I won’t even start to talk about what, up to my understanding is not meant to be a model in the first place. Fire for example. Fire is an effect, or GFX. If Minecraft contained GFX that looked like this, nobody would even think about pretending that it could be represented by a static model file. Same goes for water, obviously. But I can feel with them all too well, if you create a new system you want everything combined under its hood. The fun thing is, it still can’t be.

If you look at the header picture you can see an extract out of the file list.

1.8 contains 1809 separate files concerning models of which most have a file size smaller than 1KB.

This isn’t limited to the pseudo-GFX-models, even more or less static blocks require an horrendous amount of copy-paste use to get working. Note that I’m talking about 1.8 here, 1.9 is a whole different category of horror. I’m also not mentioning Forge, this is only about one terrible design choice that’s been made on Mojang’s side.

First we should have a look at what JSON even is. Taken from Wikipeda:

JSON or JavaScript Object Notation, is an open standard format that uses human-readable text to transmit data objects consisting of attribute–value pairs. It is used primarily to transmit data between a server and web application, as an alternative to XML.

Yes, JSON comes from JavaScript, but that doesn’t mean that it can’t be used in a lot of other cases. It’s common and easy to parse. JSON is used for a lot of web APIs for example, such as GitHub’s.

We’ve seen a lot of good uses of JSON in Minecraft. One of them is the tellraw command, or the .mcmeta files that are used for sounds and texture animation. Then why not on models as well? or so they must have thought.

One sole reason. Vertex data isn’t attribute-value pairs. Vertex data isn’t human readable. You can put it in any order you want, you will still need an editor to create models that are more than a few faces or your are wasting your time. This eliminates the use that JSON has, that you can edit those files without needing special software. Despite, who wants to edit 1809 files in order to get some visual effect? Nobody, right?

// Try 1810 to make vertex data more human readable
       __________________
      /\  ______________ \
     /::\ \ZZZZZZZZZZZZ/\ \
    /:/\.\ \        /:/\:\ \
   /:/Z/\:\ \      /:/Z/\:\ \
  /:/Z/__\:\ \____/:/Z/  \:\ \
 /:/Z/____\:\ \___\/Z/    \:\ \
 \:\ \ZZZZZ\:\ \ZZ/\ \     \:\ \
  \:\ \     \:\ \ \:\ \     \:\ \
   \:\ \     \:\ \_\;\_\_____\;\ \
    \:\ \     \:\_________________\
     \:\ \    /:/ZZZZZZZZZZZZZZZZZ/
      \:\ \  /:/Z/    \:\ \  /:/Z/
       \:\ \/:/Z/      \:\ \/:/Z/
        \:\/:/Z/________\;\/:/Z/
         \::/Z/_______itz__\/Z/
          \/ZZZZZZZZZZZZZZZZZ/
          
// Oh look, it's a cube

And this is what we have binary files for. Why would anybody encode an image in JSON? It doesn’t make any sense to do so, and it makes equally little sense to do the same thing with vertex data. A standard rule of thumb is that you should include an editor if you introduce a new file format. Most of the time it’s even the other way around. A different way of handling things would be using one of the accepted file formats out there, like Wavefront Object or 3DS Max. We want artists to use this, remember? They are used to their tools, and having our format supported by them is a huge advantage. Many of those also have their headers and are extensible so that you can add special information needed for the engine that wouldn’t be supported natively.

Is there any reason why there are that many files model files?

Yes, it’s called a permutation. That means for every tiny change there is, a new file is created. It’s interesting to look at because we now know that there are 954 different ways a block can look like4. But apart from that, I don’t see a lot of advantages. Those 1809 files have to be read from the disk. They have to be parsed. One could argue that this is faster as you could consider this a cache. But is it of any use to cache every possible rotation of a torch that consists of 6 faces? Also, 1.8 isn’t faster for me, and many others that I’ve asked.

Most engines get around that problem by dividing a model into several parts and applying transformations to them. Such a state would be called frame and the transformation from one frame into another would be an animation. There are a lot of formats out there that support frame-by-frame animations. Those could be used to replace the now still hard-coded chest animation, limb swing and others. On other occasions it makes little sense to use static frames as you directly reflect some data. An example would be filling a bucket or a tank. Here you’d use some sort of scripting language where you can simply change the position of one of the child parts.

So did we, the end user, gain anything from this? No. Did they gain anything from this? I don’t think so. Just give up on trying to use JSON here, it doesn’t lead anywhere but chaos.

4 : Wrong. There is still the dynamic counterpart that is used for chests but I only wait for them to be replaced by JSON files. Can’t take much longer as they are already planning to take over GUIs.

Why 1.9 is its own special hellscape

I have to add that there is little information on 1.9 out there yet. I wasn’t at Minecon5 so I still have to wait for the first snapshot to take a look at. However, the little I’ve seen has made me cringe. They did what I feared the most. In order to get rid of the permutation and hilariously long file list, they did the following:

{   "multipart": [
    {   "apply": { "model": "oak_fence_post" }},
    ...
    {   "when":  { "east": "true" },
        "apply": { "model": "oak_fence_side", "y": 90 }
    },
    ...
  ]
}

Sorry for the indent but this is the new standard for everything. Couldn’t find a syntax highlighter for JSON-script, I’m pretty sure that’s just me being incompetent of google search.

So this is basically the scripting language I was asking for, hooray! Except… Well, it’s still JSON.

I don’t even…

There are two thing they tell you about that you should never do. The first is premature optimization and the second one is called the “Second language phenomenon”. (I call it like that because I forgot the actual name.) It’s essentially what happens when you try to create a fluid API that feels like a language of its own. You find yourself replicating already existing methods and changing their looks slightly, adding an extra layer of abstraction that doesn’t do any good, apart from slowing down execution speed. That’s not what we have here. I don’t even know if there is a term for it at all…

Let’s get back to the term “JSON” for a second. JSON stands for Java Script Object Notation.

That’s right, it’s an object notation for a scripting language. And we are now turning that back into a scripting language.

This is so obviously wrong that I don’t want to go into detail here. Apart from that, it only suggests that there would be configurability. You can’t add rotation to a block like this. You can’t add new states to a block like this. For that you still need the block states6.

This is wasting a lot of potential. It would be entirely possible to move all of the blocks to JSON-files, allowing true configuration and a simpler way of creating simple mods. It would effectively get rid of the 3-files-problem we currently have. But please, no JSON for models. No JSON for scripting. Don’t abandon common sense…

It’s still too early to say anything more about this but… I seriously fear for the future. So here is my proposal:

Get rid of JSON-models once and forever. Use an accepted file format that is well designed.

That’s about what I wanted to say. I hope you can now see why JSON isn’t the right thing to use here.

5 : I was somewhere better, *hue* *hue*: BetterThanMinecon
6 : A block state is a state that a block has. Okay, that’s making it too simple… Say you put coal in a furnace. That furnace is now burning, it changes state. Except…