This project was inspired by a video I encountered of the game "Miku Flick". In that game, an on-screen keyboard is used to type some of the characters in the lyrics of vocaloid songs, with an intent to teach players to use the flick style keyboard. I have always liked rhythm games, and have even attempted making step files for StepMania, however in making those levels I could not fully visualize the step patterns and felt limited with four arrows, so the outcome was rather uninspired. Using the lyrics instead of arrows seemed like a perfect match - it wouldn't require much creativity on my end since the lyrics are already written, and it would help me learn different languages while playing. The concept is that some buttons are presented on the screen, each containing a word or character of the lyrics in random order; the player presses the buttons in the correct order to form the original lyrics. The song provides the word pronunciation while the button layout makes it easy to pick the associated word (compared to typing on a full keyboard) so this could be great for increasing familiarity with different languages.
In the final program, I did not implement the rhythm game functionality, because it would impose an unnecessary constraint and make step file generation more difficult. Rather, the program runs without any timing requirements, and can be played alongside a song playing in the background (or not). It acts as a keyboard where a limited set of keys may be used to type through complex texts. This means the program can be used for texts other than song lyrics (though large texts like entire book chapters may cause performance issues), or to go through a song's lyrics at one's own pace, and the step files do not require much more work than copying and formatting existing text.
A text file is loaded which contains the text to play through (such as song lyrics). The program should present an on-screen keyboard where the labels of the keys are dynamically defined such that one of the keys is the correct one to press to advance through the text. As this is intended to be a language learning tool, it is also desired to be able to load text files containing translations or pronunciations which should be displayed alongside the main text.
The generation of the dynamic keyboard is the main algorithmic challenge. Given a sequence of strings, each of which constitutes a step, it is necessary to assign each string to one of the keys or buttons. For visual continuity, only the button that is pressed during a step may change its label for the next step. An easy way to do this is to generate a queue for each button, and then iterate through the text from beginning to end loading each step string to a random queue. During gameplay, the queues are unloaded to restore the original text order. However we need to impose 2 conditions for reasonable gameplay:
Enforcing these conditions requires some creativity and could be a fun exercise for the interested reader. After implementing the scramble algorithm, there is a bit of UI work to be done to make the program scroll through and resize the text dynamically, play an animation (a green or red tint of the background) when a virtual key is pressed for visual feedback, and correctly handle keyboard focus, all done in Windows Presentation Foundation (WPF) with C#. Then the program is ready! The virtual keyboard is set up for a 3 by 3 grid, and a key may be pressed in three ways: by clicking on it with the mouse, by using the number pad digits 1 to 9 with Num Lock on, and by using the keys QWE-ASD-ZXC. The code files may be downloaded here, and the compiled executable (for .NET 7.0) with three example levels may be downloaded here.
The program loads text files in a simple format. The base file is the text to be typed on the virtual keyboard. Every line in the file represents a button label on the keyboard (except blank lines). For example, if it is desired to type by entire words, then each line is a word, or if it is desired to type by characters, then each line is a character. Associated files (translations or pronunciations) should have the same number of lines as the base file. Contiguous non-blank lines define a phrase during which the program advances the highlighted part of the text (like a karaoke mode). Blank lines indicate the next non-blank line starts a new phrase, which will scroll the visible text and hide the previous phrase. This means that in a base file which is split up by characters, an associated file with translations may include just one translation for each phrase, and the on-screen highlighting will be congruent. Consider the following schematic, with each of the three columns representing a different text file:
A Word1 Phrase1 B C D Word2 E F G Word3 Phrase2 H I
On the virtual keyboard, the letters A-I defined by the base file will appear, which the player will press in sequence. The corresponding words will be highlighted when the same line as the word is active, so Word1 relates to A-C. Phrases will be advanced when there is a line break in the base file such as between F and G, so Phrase1 relates to A-F and Word1-Word2. (Comments may also be added by starting a line with '#' which will treat the line as if it were blank.) With this approach it is possible to have foreign language characters or words in the base file, associated romanized pronunciations in file 1, and English translations in file 2, which is demonstrated in three example text step files included with the program.
I have included 3 levels with the program, each of which contains 3 files - the base file with 1 character per line, file1 with English pronunciations of an associated character, and file2 with English translations of an associated phrase. Since I don't speak these languages, the text is not guaranteed to be correct (it is based on automated online translations).
This program was a brief exercise in writing a game-like interface and applying animations and visual transitions in WPF. I think this concept, if further polished and professionally developed, could be great for helping language learners; I would readily buy a game like this. With the present software, developing text step files still requires a lot of manual effort, and the absence of timing or gradation of difficulties takes out a competitive element so there is not much of a reward after finishing a level. Automating the creation of step files from lyrics would be necessary to advance this concept, along with rewriting this using a game engine (with timing and audio features), which is all doable but not within my present time budget.