Sounds Like fun! I like the idea
I must admit our solution isn’t really any good for your web type application. What we are doing is more industrial, literally controlling robots or machines (Using Phidgets) over the internet in real time. For this our way is better as it’s a more secure peer to peer connection.
Anyway glad you found a way in the end!