Topics covered:
When working with sysex or trying to figure out that unreadable sysex implementation chart in your synthesizer's manual, it's often handy to be able to read binary numbers and to be able to read & write hexadecimal numbers. Fortunately it's not all that difficult. We'll start with decimal numbers, since understanding what really happens when we count "normally" is crucial to understanding counting in other systems.
Normally we count in a system based on the fact that we have ten fingers (or toes :-). In such a base-10 system, a digit in a number has a value which depends on the position of the digit in the number. In "5" the five means "5 units", i.e. plain "5". However, in "57" the 5 means "5 times 10", and in "572" it means "5 times 100".
So how do we parse a number like "5726"? We consider this number to mean "5 times 1000, plus 7 times 100, plus 2 times 10, plus 6 times 1". Obviously we don't really think about it this way anymore, since we're so used to dealing with base-10 numbers. Nevertheless, this is what it's all about, "behind the scenes" so to speak.
Thinking about this a bit more we see that each position in a number "adds a 0" to the meaning of a digit. Or, more formally: the value of each position is 10 times the value of the previous position. This is no wonder: since we have 10 digits to choose from (0 - 9), we run out of digits after we've used "9". We then resort to "grouping by 10's" and start over. Like this: 0, 1, 2, ... , 8, 9 -- running out of digits, so the next one is: 10 (one group of 10, and zero units). Then come 11, 12, ... , 19 -- running out of digits again, so increment the "tens" counter: 20. Et cetera, until we reach 99. Now we run out of digits again, so we want to increment the "tens" counter. But... here we run out of digits too. So to represent a group of "ten 10's" we have to perform the same trick we did when reaching 10: introduce a new position which obviously must mean "100". And there we go again: 100, 101, 102, ... You'll get the picture.
Binary numbers are very similar to decimal numbers, except that they use a base-2 system. So we now no longer have 10 numbers (0 - 9) at our disposal, but just two: 0 and 1. This is the language in which computers talk to themselves (and each other). Let's just repeat the counting exercise of the previous paragraph, this time just using 0 and 1. It starts easy: 0, 1, ... but then we've already run out of digits. OK, so we introduce a new position. Apparently we're now "grouping by 2's" (instead of grouping by 10's which we did when counting in decimal). So we get 10 (meaning 2), and then 11 (meaning 3)... but now we've run out of digits again. Well, we'll just increment the "twos" counter then. But wait... we can't increment the leading "1", since we've run out of digits there too. It's as if we've hit "99" in the decimal system -- only it happens at 3 already! Bummer... Oh well, that means we have to introduce yet another position, so we get 100 (meaning 4). Continuing we get (decimal in parentheses) 101 (5), 110 (6), 111 (7), and then we run out of digits again (like reaching "999" in decimal). By now you probably know what to do: introduce a new position, and continue counting: 1000 (8), 1001 (9), 1010 (10), et cetera.
So we see that (bin=dec) 1 = 1, 10 = 2, 100 = 4, 1000 = 8, ... Or: the 1st position counts the number of 2^0's (^ meaning "to the power"), the 2nd position is the number of 2^1's, the 3rd position is the number of 2^2's, then the 2^3's, ... This is just like the base-10 system, where we count in groups of 10^0 (=1), 10^1 (=10), 10^2 (=100), 10^3 (=1000), ... only this time we use 2 instead of 10 as the base.
Converting from binary to decimal is now a simple matter of determining what each position means, and then checking how many "items" we have of that value. Let's take binary 10011010 as an example. Going back to front, the 8 positions mean 1, 2, 4, 8, 16, 32, 64, 128 (2^0 - 2^7) respectively. We have zero 1's, one "2", no 4's, one "8", one "16", etc. Adding it all up we get 2 + 8 + 16 + 128 = 154. So "binary 10011010" = "decimal 154".
The other way around, from decimal to binary, isn't very hard either. If we had a large pile of items that we wanted to count, we could (clumsily) do that by first determining how many groups of (say) 100 we could take from the pile. Then we would take as many groups of 10 as possible from the remaining pile. Anything that's still left then would be "units". We do the same in binary, also working from left to right (as opposed to the order we used when converting binary to decimal).
Suppose we have the decimal number 214 and we want to convert it to binary. The largest power of 2 that "fits" in 214 is 128 (the next larger one being 256 which is too big). So we have one "128" and we have 214 - 128 = 86 left. We repeat the trick: the largest power of 2 fitting in 86 is 64, so we have one "64". And 86 - 64 = 22 left. Now the next smaller power of 2, 32, doesn't fit, so we have zero "32"s. However 16 does fit, giving a remainder of 22 - 16 = 6. The next power of 2, 8, again doesn't fit, but 4 does (giving 6 - 4 = 2), and then 2 does, after which we'll have 2 - 2 = 0 left. So, since we have to go all the way down to the unit-level, we conclude with zero 1's. Now we can finally write our binary number down, left to right: 1*128, 1*64, 0*32, 1*16, 0*8, 1*4, 1*2, 0*1 gives binary 11010110.
That's all there is to it. Of course you can also do addition, multiplication and any other arithmetic on binary numbers, but for now we're merely concerned with the representation of numbers and not with arithmetic, so we'll stop here.
Exercises:
Convert 12345 (dec) to binary. Answer: 11 0000 0011 1001
Convert 11 0010 0110 (bin) to decimal. Answer: 806
If you've carefully read the foregoing section on binary numbers, you'll have an easy time understanding hexadecimal. In a hexadecimal system (often called "hex"), we work in a base-16 system. The first problem we run into is that we don't have enough symbols to represent 16 different digits. Remember, in base-10 we counted 0, 1, .., 9 and then ran out of digits, so we resorted to "shift to new position and start all over", counting 10, 11, ... In hex however, we need to be able to count to 15 before running out of digits and being forced to "shift to new position and start all over". We thus need 6 new symbols to represent decimal 10 - 15. For that we simply use the letters A, B, C, D, E and F (upper- or lowercase).
So now we can count to 15, using single digit numbers: 0, 1, 2, ... , 9, A, B, C, D, E, F. Then (you'll get it by now) we run out of digits, and so introduce a new position, making "groups of 16's". So 10 now means "16 decimal", 11 is "17 decimal", and so forth. We thus have: 10, 11, 12, ... , 1E, 1F (one group of 16 and 15 units = 31 decimal). Running out of digits again, and so we increment the "16"s counter, giving 20 (32 decimal). We can continue to do this until we reach FF, meaning "15 groups of 16 and 15 units, or 255 decimal). Creating a new position gives 100 (256 decimal = 16^2).
In decimal each position meant (from right to left) 10^0, 10^1, 10^2, ... In binary we had 2^0, 2^1, 2^2, ... Clearly in hex respective positions will mean 16^0, 16^1, 16^2, etc. Converting from hex to decimal is now trivial. We'll convert 1AE3 to decimal as an example. Working from right to left we find we have 3 units (3 decimal). Then we find 14 (E=14, right?) groups of 16: 14*16 = 224. Next we have 10 ('A') groups of 16^2, or 10*256 = 2560. And finally we have one group of 16^3, or 1*4096. Adding them all up gives 3 + 224 + 2560 + 4096 = 6883.
As with binary, when converting from decimal to hex we work from left to right. First we determine the largest power of 16 that "fits" in our number and we determine how many times it fits (which in binary was easy since it was always 0 or 1). We subtract what we find and continue with the remainder. Let's try to write 10715 (decimal) as a hexadecimal number.
The largest power of 16 that fits in 10715 is 16^3 = 4096. It fits 2 times (2*4096 = 8192), giving a remainder of 10715 - 8192 = 2523. So our hex number now looks like 2xxx (the x's still have to be figured out). The next smaller power of 16 is 256. Dividing 2523 by 256 shows that 9 groups of 256 fit in 2523. Then 2523 - 9*256 = 219 is our new remainder. Our hex number is now 29xx. Continuing we see that 13 groups of 16 fit in 219, and 219 - 13*16 = 11 is the remainder. Since 13 is written as 'D', we now have 29Dx. We have 11 units left, which is 'B' in hex, and thus the final position will be 'B'. And so we find that 10715 (dec) = 29DB (hex).
Exercises:
Convert 23456 (dec) to hexadecimal. Answer: 5BA0
Convert C62F (hex) to decimal. Answer: 50735
Computers use binary counting. One 0 or 1 is called a 'bit', and a group of 8 bits is called a 'byte'. That means that using only one byte we can count from 0 (0000 0000 binary) up to 255 (1111 1111 binary). Translating this to the (shorter and more readable) hexadecimal system, we count from 00 to FF. As you'll notice one byte exactly fits in two hex-digits (remember: after FF comes 100 which uses 3 digits instead). Or: 4 bits fit exactly in one hex-digit (since 1111 (bin) = F (hex)). Therefore numbers in computer-speak are often represented as 2-digit hex numbers. If we write a binary numbers as "full bytes" (adding leading 0's if necessary), then we can divide each (8-digit) byte in 2 groups of 4 digits, and translate each group to hex separately. So taking the byte 0110 1011, we would translate that to hex as 6B, since "0110" is 6 (0+4+2+0) and "1011" is 11 (8+0+2+1) which is 'B' in hex. A no-brainer really, once you get the hang of it.
Multi-byte numbers are treated the same way. Taking binary 1001 0011 1101 1111 (2 bytes), we get hex 93 DF. Notice the spacing between each 2 digits: every group of 2 is one byte, and spacing numbers this way just makes interpretation and reading a bit easier.
Clearly we can also convert from hex to binary in a similar way. However, this is rarely needed, so I'll leave that as an exercise to the reader.
We saw that using single bytes, we can count from 0 to 255. If we need to count beyond 255, we need to start a new byte. Using 2 bytes we can count up to 65535.
Aside: in hex 2 'full' bytes are FF FF, which is FF groups of 256, plus FF units -- i.e. 255*256 + 255 = 65535. Note how in fact I used a base-256 system in the previous sentence, which in cases like this is more convenient than a base-16 system. Using "pure" base-16, FF FF would be parsed as 15*4096 + 15*256 + 15*16 + 15, which is cumbersome.
With our normal 'left to right' reading order, it's clear what (hex) 'A3 4F' is supposed to mean: A3 is the byte with "the big values" and 4F is the byte with "the small values". This first big-values-byte is called the Most Significant Byte (MSB), and the last one is called the Least Significant Byte (LSB). Rather obvious names, really.
So as long as we're dealing with relatively small values, we need just one byte, and everything's perfectly clear. As soon as we need bigger 2-byte values, things get a bit more complicated. The problem is that while we would normally write these numbers in a MSB-LSB order, computers and synthesizers don't necessarily do so. Sometimes a machine wants to receive the LSB before the MSB. So our 2-byte number 'A3 4F' would then become '4F A3', swapping the MSB and LSB.
Looking at the sysex implementation chart of my Korg M1 synthesizer for example, I see that the 'parameter change' message indeed specifies "... <bla bla message header, etc> ... LSB MSB ..." -- the reversed order! Moreover, if I want to change a parameter whose range is 0 - 99 (e.g. attack time), one would be tempted to think that 1 byte suffices. However, there may also be parameters having a range of 0 - 500 (like reverb time), where 2 bytes are needed. Therefore I apparently always have to send 2 "value bytes", making the MSB = 0 where applicable. So when sending a value of '2', I would actually send "... 02 00 ...": a LSB of 2 and a MSB of 0.
Things are further complicated by the fact that in sysex we're not allowed to use the full 8-bits of a byte. That's what the next section discusses.
In the MIDI protocol, a distinction is made between so called "status bytes" and "data bytes". A status byte tells the synth (or whatever machine) that "here comes a message of kind X". Like "here's a note-on message coming", or "what follows is a sysex message". Following a status byte, a couple of data bytes are sent, specifying e.g. which note should be switched on, and how hard the key was struck (velocity). (Only the so-called System Realtime Messages have no data bytes attached to them, all other status bytes require at least 1 data byte. See e.g. the Usenet MIDI Primer.)
Status and data bytes are distinguished by their 'top bit' - the most significant bit. In a status byte, the top-bit is always 1, whereas in a data byte it is 0. So a status byte looks like (binary) "1xxx xxxx" and a data byte looks like "0xxx xxxx". Very clear and very simple. However, the immediate consequence of this practice is that we now have only seven bits per byte left to "put data in". I.e. we can only represent numbers 0 - 127 (hex 00 - 7F) with one byte, instead of 0 - 255! And so as soon as we need to represent numbers 128 and up, we need 2 bytes for that.
In practice this isn't very hard: we just use up our 7 bits, and after that continue counting in the next byte. In a 2-byte MSB-LSB system, the number 127 would be (hex) 00 7F and 128 would be 01 00. (On a LSB-MSB system this would of course be '7F 00' and '00 01' respectively.) We can now represent all numbers from 0 to 16383: the biggest number we can make is 7F 7F, which would be 7F groups of 128, plus 7F units, or 127*128 + 127. (Or, calculated differently, we have 14 bits available, giving a range from 0 to 2^14 - 1.) Enough for most purposes.
So now you want to send you synth the message that the reverb time should be set to 415 ms. The value to send is 415, the synth uses MSB-LSB ordering, and you have to figure out the hex code for that... Well, 415/128 = 3.24, so we've got 3 packets of 128. The MSB is thus 03 (hex). Then 415 - 3*128 = 31, and 31 = 16 + 15, which gives hex 1F. The entire 2-byte number thus becomes "03 1F". An easier way to get the same result might be to consult the conversion table.
We now know how to convert numbers 0 - 16383 to "2-byte 7-bit hex" notation. But... some parameters on your synth also use negative values! Things like Filter Cutoff, or Velocity Sensitivity may have a range from e.g. -99 to 99. How to represent those?
Again the idea is rather simple: we simply count backwards from 0, still using our 2 7-bit bytes. I.e. we use a sort of "wrap around" method to get to the negatives. If we have a range from (MSB-LSB) 00 00 to 7F 7F, then what is the first number preceding 00 00, using "wrap around"? Exactly: 7F 7F! So now 7F 7F means "-1". But the consequence is that 7F 7F can't represent 16383 anymore: one number representing 2 values would clearly be a mess!
What we do is: start at 0, and simultaneously count forward and backward. Somewhere "in the middle" (at (hex) 40 00) the two sequences will meet, and that's where we stop.
So, counting forward the old-fashioned way we get: 00 00 (0) 00 01 (1), ... , 00 7F (127), 01 00 (128), ... 01 7F (255), 02 00 (256), ... , 3F 7F (8191).
And counting backward gives 7F 7F (-1), 7F 7E (-2), 7F 7D (-3), ... , 7F 00 (-128), 7E 7F (-129), 7E 7E (-130), ... , 40 00 (-8192).
We already covered translating positive decimal numbers to 7-bit bytes. To do the same for negative numbers, proceed as follows: take the positive value and subtract it from 16384 (2^14). Convert the result to hex. So a value of -99 would be treated thus: 16384 - 99 = 16285. Translating 16285 to hex gives 7F 1D (127*128 + 29*1). Putting it all in order we have:
dec - hex (MSB-LSB)
---------------------
-8192 - 40 00
... - ...
-131 - 7E 7D
-130 - 7E 7E
-129 - 7E 7F
-128 - 7F 00
... - ...
-4 - 7F 7C
-3 - 7F 7D
-2 - 7F 7E
-1 - 7F 7F
0 - 00 00
1 - 00 01
2 - 00 02
... - ...
127 - 00 7F
128 - 01 01
129 - 01 02
... - ...
8191 - 3F 7F
If you've come to this point, you know everything you ever need to know about number representations for MIDI use.
(c) H.J. Veenstra 2001.