In prior installments I described the process that led me to implementing a piano sheet music reader on a Raspberry Pi. In this installment I will show you the working version, provide something of a manual for operation, or specification for what it does. If you want the actual code it will end up on Github (still working on that as of this writing).
Let’s start with the Library in Calibre and getting sheet music into it. First, a note on Copyright – I purchased and owned all of the music I scanned. I am not advocating this as a mechanism for “sharing” music that is copyrighted.
Here is how I got sheet music into Calibre:
- Scan each page as a grey TIFF to disk on windows. I used 400 DPI mostly, though 300 seemed adequate. Much more and the pages are quite large (and may slow down display on the Pi). My scanner could do dual side scanning, but not large sheets. For smaller books I cut the spine off and fed them in as sheets in the document feeder. For larger sheets and larger books I had to scan them one by one. Quite tedious. Sometimes quite hard to align a large sheet so as not to cut off portions.
- Once on disk as a TIFF, I ran through Photoshop. I found that the FIRST thing I needed in Photoshop was to convert from indexed color to true grey. My scanner was outputting “indexed color” for its own “Grey” and Photoshop does a lousy job of editing indexed color (especially in terms of straightening — it produced very jagged lines). Experiment with your own and what format it uses.
- I cleaned up any trash (old handwritten notes, scan artifacts), straightened the sheet, and cropped it TIGHT to the music. Large margins just make for smaller notes when displayed.
- I then used Automate, PDF Presentation, and generated a multi-page PDF. I set it to no text, compressed, 8 bit and no downsampling.
- Save this as a file name of title – author.pdf. That is a standard format for Calibre for import and will automatically populate the title/author fields. It and Windows are both OK with parentheses and commas and most other punctuation, though not exclamation marks. If you find issues with such special characters, add them later in Calibre. I adopted the idea of putting a play or movie (where relevant) in parentheses in the title, and later manually added books or collections as a series.
- I set up Calibre so it auto-added and tagged each new addition with a tag of “Newsheet”. This way I could drag and drop dozens of songs at once into Calibre, then go back and work on all of them – add series names, category or other tags. Each “book” I tag with “Music” so that I can isolate them from real books; the MusicalPi application will only select items tagged “Music”.
So I’m off and running with a building library. Very tedious to build, but very doable. Now when the program starts, I get a library display: This display scrolls if you drag on it, so you can scroll down to any title you like. You can also click on the heading to re-sort by that field, e.g. by Collection (i.e. “Series” in Calibre).
For large catalogs this is still tedious, so there is a search bar. Touch it, and the on-screen keyboard comes up. As you type letters it finds that string anywhere in the line and shows the result as you type. Notice here I’ve touched the search box, and started typing the first few letters of Christmas.
This could be made more complex with individual searches on each field, but so far this seems to work fine. I do think in a future version I will put individual “Playlists” so people can keep the songs on which they are working, or performing, and call them back as a group.
Once you get to a song, touching it brings you to the Play-Review screen. This screen has white borders, and a menu bar. It is meant to look through a piece (especially a longish one). You can also use this screen to change to other “up” sizes. By default it starts with two pages shown, or “two up”. You can also change to other sizes, for example here is one with 4×2 Up, clearly not usable for playing, but fast for navigating to find a particular passage. You might also use this on a portrait oriented monitor to display just one page at a time, if you needed very large notes (perhaps someone with decreased vision).
Once you find where you want to be, and the format (how many “up”) you touch “Play” and it shifts into a black background, de-clutters the screen, and is ready for you to change pages. At present, though I am not all that happy with the appearance, it first shows you the areas on the screen that respond (in Play mode) to touches to change page, or (at the top) to return to the menu:These colored areas display a couple seconds and go away, leaving just the two sheets displayed.
Now comes the complicated part – how to turn pages. Going forward is pretty easy, though not at first obvious. I wanted there to be plenty of overlap – showing the page you are on, while also allowing you to see the next page. This way you can play to the very end and still have the next page ready.
So going forward, this is how it pages. Let’s say you are on pages 1 and 2, with 1 on the left of course, and 2 on the right. You touch the pedal or the right side of the screen and this happens:
The LEFT page, which is page 1, is replaced by page 3. The right page you are playing, 2, stays there. Notice in the above the green border – that is a brief flash to show the page that just appeared. A few seconds later (as you complete playing page 2 and shift to page 3 on the left), the right page then changes to page 4. Much as though you turned a page in a book.
This works very nicely going forward in a continuous fashion. It is very natural and simple, like turning pages in a book, except you get to keep seeing the page you just turned even as it shifts to the other side.
But unfortunately music does not just go forward, it goes backwards as well. The program almost exactly reverses the process, except instead of changing by 2 pages at a time (where page 1 as replaced with 3) it goes back one at a time. Let’s say you had 3 and 4 displayed, and needed to repeat a section that began on page 2. Step on the Back pedal (or touch the left side of the screen) and page 2 appears on the left, and a few seconds later page 3 appears on the right replacing 4, and keeping things in order.
Need to repeat back to page 1 not 2? Step on the pedal again (no need to wait for a complete cycle), it goes 2 pages back – page 1 replaces 3 on the left, and a few seconds later page 4 cycles.
This does not work well if you are playing on the left page and need to go back; it immediately removes the page you are playing. You have to get used to that. I have some ideas for adjusting it, but doing it this way is no worse than paper, and faster (and it does not fall on the floor).
Before leaving this subject it is worth showing how it works with one page – it does a half page at a time, letting you finish the bottom but replacing the top.
Again, works great if you are going continuously forward, but is a bit confusing if you are going backwards — but with only one pane what can you do? This appears after a few seconds. It might be useful if someone wanted to run this on a tablet, or needed a portrait oriented very large screen for
limited vision.
On these sheets you can see some mistakes – the program is putting up a green page number, but as some of these scanned from books, they show a printed page number as well (in black) that is different. As I scanned more pages, I explicitly started removing page numbers from the scanned pages, and let the MusicalPi program number them instead. Makes much more sense to someone viewing them, rather than starting on page 321.
While on the subject of display, a few things I did in code that are worth mentioning:
- When the PDF display is generated, it must be scaled. The program very carefully calculates the scale needed for the full play mode (without the menus). To run faster, it starts building these as soon as you select a piece, even though it is going to display smaller versions for review. It scales those on the fly (scaling the image itself). This gives a somewhat less high quality display for review mode, but prevents re-generating the pages for play mode. In play node they are at the highest quality.
- The program also tries very hard to use ever possible pixel high and wide, while maintaining aspect. It runs in full screen mode on the Pi, so there are no menu bars (there are on some screen shots above just from the mechanics of capturing them).
- Related to the above, I find that cropping the images from a page scan to be very tight around the music allows it to display larger and more readable – just do not let the notation actually touch the edge.
Most of my recent efforts have been spent on scanning and building up my library. Soon I plan to work on new features, maybe annotations (hard), playlists by user. I am considering allowing one to program “jump to” points in a specific piece, so the music knows what page to return to as you loop through songs, and what page to go forward to (e.g. to Coda). But as I use it, I find less and less need for specialized tools there – just touching the screen is so much easier than paper I may not do more than the page turn logic I use now.
In the next section a few words on setup, hardware, building, etc. That section will not go up until I put the project on Github, so if you happen to land here and there is no next section – soon. Really. Soon.
Dear Linwood,
I just cloned everything you did – both software and hardware. Works like a charm !
The only difference on my system is that Calibre runs on a Mac, and I suffered a bit to set up the sharing since I was not that used to this kind of things. sshfs was the trick.
I still have to scan all my sheet music. Will take ages but it is worth doing it. Also, I have a to find a safe way to mount the touchscreen on my upright Pleyel piano, but this is minor stuff.
Thanks so much for sharing your code and also for your excellent “didactic” skills.
Cheers from France.