Automating Midi File Conversions for Compatibility

This is an article on Midi remapping via scripts so that manual midi editing is not needed to get EZDrummer, Guitar Pro and other midi formats to conform to the more or less General Midi assignments used by Beat Buddy, but it could be used for any midi format transformation needs. I started out thinking about converting all my loops to BB or making custom kits but since there at compatibility reasons for needing to keep originals, and ToonTrack would need a huge instrument map I decided to do only as needed but even then quickly tired of single file fiddling around. Spending even 5 mins per file adjusting notes, tracks etc. is too long to waste on a repetitive conversion. In summary I was able to use free utilities, scripting and the OSX Automator utility to set my system up so that I can right click on a file and select either a format check to look at the track layout/type or to convert it from Type 1 or Type 0 and remap notes. One click convert, no midi editing.

The steps below show how to check to see what format a midi file is, what channels are in it, what notes are assigned to what channels and to remap notes to other values.

Notes on Formats, Tracks, Channels:

Tracks in a midi file are just like a multitrack recorder. Drums,Gtr,Bass,Vocals don’t have a specific track assignment any more than a recording console has dedicated instrument channel strips until the engineer assigns them. GM does suggest instruments go on designated channels. Channel 10 should be Drums.

BeatBuddy’s midi files are Format Type 1 (separate instruments per track) with a single instrument on track 1 assigned to channel 10 - GM Drums.

EZDrummer: Format Type 0 (all instruments on one track like a master mix) They contain a single track with drums on channel 1 which is normally Piano.

Guitar Pro Midi Export: Format Type 1, can contain multiple tracks depending on the solo/mute selection. Whatever track the drums are on is assigned channel 10. Default export will create a multi-track output file. Be sure to solo the drum before exporting for a single channel 10 track.

Assuming all the above is good, then there are the note assignments. BeatBuddy notes are consistent but don’t have every type of note assignment for extra techniques etc. so they are only occupy a small range out of the 128 notes possible and conform closely with GM, except on Tom sounds.

EZDrummer and Guitar Pro exports are NOT GM note compliant+ Each needs to be validated and modified so that undefined notes aren’t in a BB midi as they will not play a sound leaving gaps and other problems with the wrong sound being played. Problem with BBM is that it doesn’t warn when non-GM midi’s are fed to it, they just don’t sound right and may just not sound at all.

There are two ways to do format, track or note translations. NOT midi editing for timing and or creation but getting the midi into a format that BB will have some chance of playing correctly.

Manual - tweaking in a DAW of every damn midi file or a program/script to do menial conversion.
Scripting - using a program to compare to a translation table and make automated appropriate changes.

I was unable to locate a modern midi util for Mac or Windows but located a set of old DOS utilities. They do exactly what is needed, but not without issues. The author is not wanting to revisit a cross compile or upgrade to get them to work on OSX. A bigger issue was that do not support long file names. The utilities are DOS 8.3 limited which means they can’t read anything like a midi named EXD-Standard@002-12-6-fill1.mid so I searched for a good way to workaround to do renames as part of the process.

The CLI utilities are freeware, there is a GUI that uses them but then, the point is automation not wasting time in a GUI to convert for compatibility. You will only need a few as in caps explained below. They can be found at: http://www.gnmidi.com/gnfreeen.htm

They all make use of command line Kung Fu parameters and don’t all run as expected so the rest of this guide shows how they can be used to make your life easier. I didn’t want to use Parallels every time I had a file to work with so found that the CLI’s work within DOSBox. If you have OSX the installer is here:
http://sourceforge.net/projects/dosbox/?source=typ_redirect

For either OSX or Windows these utils are most useful to be setup in batch or command files. The DOSBox config has an autoexec.bat section. I’ve included 4 config files and automator scripts that will each do a different set up steps to convert or document your midis. DOSBox configs go in Users/yourname/Library/Services folder. Automator services scripts can be loaded in automator to check what they are doing and modify as needed for your folder locations then saved to activate. They are Services so that is when your right click on a file can pick services, then what your OSX can now do directly to a midi. DOSBox and other dialogs will popup as they run.

The following are from the config file autoexecs that convert notes from a Guitar Pro file - Remember, already Type1, Channel 10. You’ll need to RTFM on the utils and DOSBox, Automator configs but this is a pretty complete processing script and where each CLI would come into play to preview,convert,verify.

[autoexec]

Lines in this section will be run at startup.

You can put your MOUNT lines here.

Guitar Pro is already Type 1 channel 10 for drums

mount c /Users/Norbert/MidiUtils
path z:;c:\EZXRemap
c:
cd EZXRemap\InProg
del midilog.txt
MIDIFMT inmidi.mid >>midilog.txtMIDITRK -v inmidi.mid >>midilog.txt
CHKRANGE -range BBLoHi.RNG inmidi.mid >>midilog.txt
MAPNOTES -chg EZXMapBB.CHG -c 10 inmidi.mid outmidi.mid
MIDIFMT outmidi.mid >>midilog.txt
MIDITRK -v outmidi.mid >>midilog.txt
CHKRANGE -range BBLoHi.RNG outmidi.mid >>midilog.txt
copy outmidi.mid …\Done\outmidi.mid
del *.mid
cd …\Done
exit

This one is similar but for EZDrummer converts to type 1 and remaps the channel to 10.

[autoexec]
mount c /Users/Norbert/MidiUtils
path z:;c:\EZXRemap
c:
cd EZXRemap\InProg
del midilog.txt
ren inmidi.mid intmp.mid
MIDIMAP.EXE -map 1 10 intmp.mid inmidi.mid >>midilog.txt
MIDIFMT inmidi.mid >>midilog.txt
MIDITRK -v inmidi.mid >>midilog.txt
CHKRANGE -range BBLoHi.RNG inmidi.mid >>midilog.txt
MAPNOTES -chg EZXMapBB.CHG -c 10 inmidi.mid outtmp.mid
MIDI0TO1 outtmp.mid outmidi.mid
MIDIFMT outmidi.mid >>midilog.txt
MIDITRK -v outmidi.mid >>midilog.txt
CHKRANGE -range BBLoHi.RNG outmidi.mid >>midilog.txt
copy outmidi.mid …\Done\outmidi.mid
del *.mid
cd …\Done
exit

A lot of this is output to a log file and checking before and after so it is more than you need to whack out a conversion or format check. The utilities have readme files and the site has descriptions of the various utils.

A config file above defines the range of notes that will be checked by CHKRANGE to determine if out of bound notes exist. I defined it as a simple start-end based on the standard BB kit.

This is BBLoHi.RNG:

range of instruments

programnr(0-127) [LSB-Banknr [MSB-banknr]] : minnote-maxnote

drumkit is usually MSB-bank 127

BeatBuddy standard instruments

0 0 127 : 36-59 # BeatBuddy Low-High

BTW: Defining a RNG file with like only 0 0 0 would list all notes in the file.

I came up with this magic conversion file for use by MAPNOTES above. It took some trial and error but is a pretty complete remapper. The object is to read the midi and move all notes or keep the correct ones where they are. No quantizing or other modification is being done, just remapping. On the left is what is seen when reading the input file, on the right is what it will be replaced with in the output file. The attached PDF is a spreadsheet showing the kits and midi notes. T

his is content of EZXMapBB.CHG:

; Transform to BeatBuddy

127 => 37
126 => 38
125 => 38
124 => 46
123 => 46
122 => 42
121 => 46
120 => 46
119 => 42
118 => 57
117 => 51
116 => 51
115 => 49
114 => 53
113 => 51
112 => 51
111 => 51
110 => 49
109 => 53
108 => 51
107 => 57
106 => 57
105 => 51
104 => 51
103 => 49
102 => 53
101 => 51
100 => 51
99 => 51
98 => 49
97 => 53
96 => 51
95 => 49
94 => 49
93 => 51
92 => 51
91 => 49
90 => 53
89 => 51
88 => 51
87 => 51
86 => 49
85 => 53
84 => 51
83 => 49
82 => 50
81 => 50
80 => 48
79 => 48
78 => 48
77 => 48
76 => 56
75 => 50
74 => 43
73 => 50
72 => 43
71 => 37
70 => 38
69 => 38
68 => 38
67 => 37
66 => 38
65 => 42
64 => 46
63 => 42
62 => 42
61 => 42
60 => 46
59 => 49
58 => 57
57 => 57
56 => 49
55 => 49
54 => 49
53 => 53
52 => 57
51 => 51
50 => 50
49 => 49
48 => 48
47 => 50
46 => 46
45 => 48
44 => 44
43 => 43
42 => 42
41 => 43
40 => 38
39 => 38
38 => 38
37 => 37
36 => 36
35 => 36
34 => 36
33 => 38
32 => 57
31 => 57
30 => 57
29 => 49
28 => 49
27 => 49
26 => 46
25 => 46
24 => 46
23 => 46
22 => 42
21 => 44
20 => 42
19 => 42
18 => 42
17 => 46
16 => 46
15 => 46
14 => 46
13 => 46
12 => 46
11 => 42
10 => 44
9 => 42
8 => 42
7 => 42
6 => 38
5 => 43
4 => 43
3 => 54
2 => 54
1 => 56
0 => 51

When you run the utilities the thing you want to see in the log is for CHKRANGE to come back with no notes that don’t fit in the Beatbuddy range and obviously the newly converted file in your output folder. Would be nice if at some point input validation is done by BBM when importing a midi into a song so that it checks type and range and warns of undefined notes with an opportunity to remap via templates. To Whom it may concern - Programming-101: All input must be validated, never trust a user input value or file.

Since I have Automator doing the dirty work, all I need to do is right click on the midi and OSX now understands how to work with it directly. Pretty cool feature of OSX, prob could do similar with Windows power shell etc.

[ATTACH=full]783[/ATTACH]

Behind the scenes
[ATTACH=full]784[/ATTACH]

Enjoy

Jeez thanks Norbert, you’ve sure been busy (plenty of bedtime reading there)! This program may be useful as well if someone wants a quicker way (I got a bit lost with yours I must admit) -
https://midifilemapper.codeplex.com/. It converts between GM, EZDrummer, Addictive Drums etc. but doesn’t have a ‘BeatBuddy’ conversion unfortunately. Not sure how accurate it is as haven’t tried it as yet.

Noob question here !
I have Cubase 5 and if I make a MIDI groove with Addictive Drums in Cubase would it be possible to export/import it to BB with BB’s drumsets ?

Are you asking if you can make BeatBuddy Songs with Cubase…then yes you can. However ensure you follow the drum mapping as the Beatbuddy does not support every midi note within the GM drum map.
If I have missunderstood please rephrase your question.

Thanks !

Yes make (drum) grooves in Cubase and use them in in BB.

I downloaded the BB Manager and was able to listen to all the drumsets and styles which was really cool.

Are there any drum maps specifically made for BB when using Cubase ?

Thanks for your videos, I really enjoy them !

Cool, I didn’t see that one, will take a look. The way I got it done was out of desperation, and really just to illustrate that there needs to be a programmatic way to adapt midis to make BB more user friendly to 3rd party midis and kit layouts. Groove Monkee is the only compatible loop supplier I think right now. This util looks like it uses a similar set of functions as the CLI for conversions and also uses a map definition to read and translate to new. It could be Windows only. With all mapping context is also important so general midi conversion is is assuming a standard rock kind of kit is the output. A different translation table would be needed for a latin/percussion that probably would be way different to translate to. Another good reason to avoid making custom kits that don’t use some kind of standard layout to be compatible with. In the map definition above you can see the tom selections I made from EZDrummer to BB so probably would be very close to GM with maybe a few toms flipped. Thanks for the tip! If it works ok I’ll make a BB file for it and post one. Main takeaway from the CLI utils is that they (or similar utils) can be used to determine what the midi file contents are and can correct it before BBM use without midi editing.

Windows only so I don’t want to need to start Parallels on my Mac to use but the source code section for the util has the XMLs used for mapping. All the converts to GM are close to BB. This would be a EZDtoBB.XML convert that it could use, was nice to see the maps I did above are looking like the right ones that someone else also got. I’ve made a few edits to their EZD to GM file to swap the toms below. EZX drums use multiple notes per kit parts so three different notes could be found in a file for a Kick for example where BB needs it on one = #36. Their XML’s can also be translated to line by line values and used for the above scripts. It also a good referenced for EZX Latin etc. needing different mappings. OSX version anyone?

https://midifilemapper.codeplex.com/SourceControl/latest#MidiFileMapper/Maps/EZD to GM.xml

Would it be possible to make one with “Addicted Drums 2” to BB ?

I’ve attached the keymap for “Addictive Drums 2”

Too bad there’s no AD to GM on the utility site, would be easier. Using the XML above you’d just need to list out all AD notes in one column then the new BB place they need to go per the instrument name in the other.

Is it hard to do ?

How do we use your EZDrummer to BeatBuddy ‘midi mapping rules’ in Midi File Mapper Norbert? I presume save it as an XML file for starters but then what?

Yes it would be a copy/paste to replace that section of one of their XML files or modify their GM one with the toms that were changed. Sorry but I have a Mac so haven’t got the file mapper util loaded. The XML’s they have look like they go into a config folder when it is installed so that new ones can be added. I just modified their GM file for what I did for BB Toms.

Thanks mate. I got it working but think maybe some other notes need switching around in the XML file as it’s not converting correctly (for example “” should be OutNote='38" /> as BB doesn’t have note 40). No time now but will have a play around with it sometime.

whoops, I didn’t read it over to thoroughly but that sounds right for #38.

I think my head is going to explode:confused:

Yup, felt the same way when doing the above :slight_smile: Depending on where your midi patterns are coming from there can definately be a lot to a little work needed. These links might help if you use Logic or are using Guitar Pro.
http://mybeatbuddy.com/forum/index.php?threads/apples-logic-daw-can-remap-to-bb.2817/
http://mybeatbuddy.com/forum/index.php?threads/import-midi-from-guitar-pro-6-to-bbm.2792/

Thanks for your information. It seems you succeeded installing “Midi File Mapper” on your PC. Mine is pure Windows8 but I get an error running it on processing midi files, nothing is converted even if installing process looks to run well. How did you make it running ok on your PC ?

Hi, I cannot find the utils any more…
I only see a demo version of GNMIDI…

Not sure that the utilities are hosted by this forum.

Hi, I don’t think they are…
Couldn´t find them anywhere. Tried to Batchconvert with the demo version of GNMIDI, but it didn´t work, due to licensing requirements.
Mmmmmhhh…
Any chance to give an hint (or a .zip) :wink: with everything needed, or where somewhere else to find?
thank you

Or would it be possible to just do a map for Midi File Mapper?
https://midifilemapper.codeplex.com

Here would be the maps already available
https://midifilemapper.codeplex.com/SourceControl/latest