Cyanogen has ported my multi-touch code to the Android 2.0 / Eclair API, enabling much simpler implementation of multi-touch apps on top of the new Eclair multi-touch api. If you’re implementing MT apps, trust me, you don’t want to do it without this Java class 🙂 The Eclair build includes both the official Linux kernel API for multi-touch, as well as changes to the MotionEvent class to support multiple touch points. You can use these directly to implement multi-touch apps on Android without any additional code, but dealing with the huge amount of noise in the event stream on touch up / touch down events is a hard problem. And re-implementing the boilerplate code for pinch-zoom over and over again is pointless, because it’s not as straightforward as you might think (you have to transform back and forth between two different coordinate spaces, etc.). Anyway my code helps with those problems, and to easily implement multi-touch apps on Eclair all you need is the patched version of my MultiTouchController.java class (see update below). (This is the class used in cyanogenmod to implement multi-touch scaling in the browser and the gallery.) This class dramatically simplifies the logic necessary to implement dual-touch scaling (pinch zoom) as well as other dual-touch operations involving the distance between the touch points and their orientation. There is also a *lot* of finicky behavior on current capacitive touchscreens on touch up / touch down events (e.g. one axis but not the other axis will suddenly jump to an ordinate of zero while the coordinate still reflects the correct location). This code takes care of cleaning up the event stream pretty dramatically so you get stable and useful dual-touch information, has lots of useful helper methods and classes, and has a high-speed integer sqrt for calculating the distance between the touch points. Anyway, thanks for porting the code, cyanogen [if anyone’s paying attention, it means cyanogenmod is basically now Eclair]. I hope this Java class is useful to somebody — please drop me a line if you use it to implement multi-touch in your own projects. UPDATED [2010-06-09]: Updated code to Android-2.2, added support for 3+ touch points, and moved to hosting on Google Code, see the link below.
MT Controller: UPDATE [2010-06-09]: get it from the new Google Code project here.
MT Demos (should work out-of-the-box on an unpatched Droid or Nexus One): MTVisualizer.apk | MTMapsDemo.apk | MTPhotoSortrDemo.apk Note that MTVisualizer is also posted as a free app in the Android Market.
MT Demos, source code: MTDemos.zip
UPDATED [2010-06-09]: List of applications that make use of the MT Controller — see these for more examples of how to incorporate the MT Controller class into your own code:
- Mickael Despesse’s “Face Frenzy” face deformation app (not yet on the Market)
- Yuan Chin’s fork of ADW Launcher to support multitouch — Yuan says, “I just made use of the backported version by mmin of your MultiTouchController in my fork of ADW Launcher (to implement the pinch zoom), and I have to say it really simplified things (on top of the backwards compatibility)! Thank you! By the way, I will let @anderwebs know about it soon so it will be included in the ADW Launcher in the Market too!”
- David Byrne’s fractal viewing app Fractoid, available free in the Android Market and with source available under the GPL. David is the first person who has emailed me to say he’s using my code in an actual app shipping in the Market, thanks David! Fractoid is really nicely done, the code is clean, and the pinch-zooming works really well. It’s probably the best example of pinch-zoom out there right now because it zooms to exactly the pinched size, unlike the browser that can only zoom in-out by certain increments and constantly re-flows when you’re zooming. Try out Fractoid and let David know what you think!
- mmin’s handyCalc calculator, for pinch-zooming in/out of graphs.
- Formerly: The browser in the cyanogenmod replacement firmware (and before that, JesusFreke) — also possibly other firmwares like dwang5. Now replaced with official pinch/zoom in the second Nexus One OTA for Android 2.1.
UPDATE [2010-02-02]: Google releases an OTA update for the Nexus One that includes pinch-zoom in the three apps that make sense (Browser, Maps, Gallery3D). Unfortunately however, I looked at the multitouch code used in these apps with baksmali — and it appears that all three MT controller implementations are different. Also they don’t zoom around the correct point (the center of the pinch). My multitouch controller does the correct transformation between screen coordinates and object coordinates to get the center of the zoom correct, and makes writing apps like this much easier — they could have saved themselves some time and work 🙂
—-
— Please donate if you use and like multi-touch on cyanogenmod (or previously on the JesusFreke ROMs) — it will encourage me to keep working on cool stuff!
would you please post a how-to for beginners who are completely unfamiliar with this please?
This code is intended for programmers to use in creating their own multitouch apps that need pinch-zoom functionality. Are you a programmer, or are you just looking to use pinch-zoom in applications on your phone? If you’re a programmer then I could post a tutorial.
James — I just posted the source code to some demos that at least show you how to use this multi-touch controller class, let me know if you are interested in how to code multi-touch applications if the demo code is not self-explanatory enough.
Hi Luke,
I am working on MT support in Android an your work has been really helpful. I was experimenting with wanting to build MTVisualizer and other MT apps of yours for Nexus. I am fairly a newbie to this. The controller java file you have linked here and the code from your earlier cupcake support did not help. I am sure I am doing something wrong. I tried to find where Cyanogen has the complete MTVisualizer package but couldn’t find it. Any ideas how I should proceed.
Hi Bhupesh,
You’re not the first one to ask. Let me put together a port of the demo apps to Android-2.x, complete with Eclipse projects so you can easily build and start hacking, and I’ll put the port up here. Hopefully won’t take me long as the code should just compile out of the box.
Luke
Thanks Luke. Greatly appreciated.
Bhupesh — I just posted working code and demo .apks as an update to this post.
Luke, you are quite easily one of my heros. Nice work. 🙂
Ditto to the complements. Thank you so much Luke. Greatly appreciated.
can you include this library directly in your app so it works on phones w/o cyanogenmod? Anyone have advice for someone who wants to have pinch zoom in their app working from 1.5-2.1 Should I release multiple apps or will this do the trick?
The answer is yes you can use this without cyanogenmod for Android 2.0+, and no you can’t use it without cyanogenmod (or equivalent) for android 1.0-1.6. cyanogenmod contains the necessary kernel patches to get this working on Android 1.0-1.6. The Android Open Source Project contains all the changes you need to get this working on Android 2.0+ without rooting and without any changes to the operating system. (In fact you don’t even need my MultiTouchController, it just makes things easier, and/or shows you what you need to do to implement your own controller.) This particular controller no longer works with the old cyanogenmod / JesusFreke kernels, because those used a hack to get multi-touch working. That hack is not needed in Android-2.0+, i.e. on the Droid or Nexus One (or later phone).
[…] Hutchison has ported his multitouch controller to Eclair and Cyanogen has released a modified browser that implements pinch zoom. These are the […]
[…] has stepped up and provided a temporary hack while we wait on Google. Luke Hutchison has ported his multitouch controller to Eclair and Cyanogen has released a modified browser that implements pinch zoom. These are the […]
[…] Hutchison has ported his multitouch controller to Eclair and Cyanogen has released a modified browser that implements pinch zoom. These are the […]
Is this the code running on CM 4.2.13?? If so, it’s a dramatic improvement over previous MT implementations. Nice job!
Brilliant. Thanks for the hard work.
[…] Hutchison has ported his multitouch controller to Eclair and Cyanogen has released a modified browser that implements pinch zoom. These are the […]
[…] Hutchison has ported his multitouch controller to Eclair and Cyanogen has released a modified browser that implements pinch zoom. These are the […]
Hey Luke,
really thanks for your multi touch controller. I would need your help. I was using the multitouchcontroller.java for some time using cynogen 4.2.3.1 and it works really fine. However when i upgrade to 4.2.3.13 the multitouch class no longer work. I understand that you have ported to eclair and it seems that when i use your new multitouchcontroller.java there are some event methods that are only available for android 2.0. Hmm but I was wondering, cynogen is still using 1.6 for his newest release and yet he can implement multitouch on browser. Is there anything I’m missing out? hope to get my custom multitouch application back 🙂 thx thx
Glad you’re using the multitouch controller class! I don’t know of many people that are actually using it yet. Yes, you are right about the new methods introduced in Eclair. I don’t know at which point cyanogen pulled out the old kernel hack, but it’s likely that if you switch to using the newer MultiTouchController.java class posted on this page, your code will magically start working again (it uses the Eclair API now). I won’t be supporting the old kernel hack anymore, the Eclair API is the way of the future. Thanks so much for your interest! 多谢!
Nice inspiring blog, dude. I enjoy your posting. I am about to create blog about mobile phones, too.
Thanks!
Hey, did you try to submit this abstraction layer upstream?
It looks really nice, it would be interesting if this became part of AOSP and came preinstalled on future versions of Android.
Thanks for your work!
I didn’t; I suspect that upstream won’t be friendly to this until whatever is holding them back from aggressively supporting multitouch in all countries is resolved.
Doesn’t AOSP already feature a multitouch API?
AFAIK the only thing they are doing is shipping devices without apps implementing multitouch (in the US), in spite of multitouch support being already there.
If you have developed such a nice API which simplifies the existing API and just goes on top of it, with the only goal of simplifying things for developers, I doubt they will refuse it for that reason 🙂
I do not wish to pressure you into submitting it if you don’t want to (since its your code after all 🙂 but I must tell you I would be very happy if you tried (since you can not lose anything).
Anyways, thanks again for your great contribution, and good luck in case you decide to submit!
Yes, AOSP already features a multitouch API. However it is not supported in any out-of-the-box configurations within the US for unknown reasons. Upstream are dragging their feet on multitouch (basically it’s not a priority) until they can happily support it in all regions.
My API only simplifies pinch-zoom. Though, realistically, that is the only operation that is really useful with current sensor technologies (other than a few MT gestures maybe), because of the limitation where coordinates snap if they are close together in X or Y. Furthermore my controller is good but not the be-all and end-all of multi-touch, it just really helps when writing code to deal with pinch-zoom so I hope people will use it rather than re-inventing the wheel.
You can actually submit a feature req to the public Android issue tracker yourself if you want, I would love to see my code included — I just haven’t had much luck at all with that issue tracker and am not interested in using up time trying to convince the Android team my code is useful and should be included when I know they have a long list of internal priorities that are higher on their TODO lists than most of what’s in the public bugtracker. (They informed me when I enquired about the public tracker that almost everything there is a dup of some internal bug that they already know about, so they don’t give it much weight.)
Thanks for the support and encouragement though. I am certainly interested in seeing multitouch move forward on Android, and I’m glad there’s now an official API — that changes everything.
[…] fix (thks jnwhite) – Autorotate home screen – Multitouch in the browser (using Luke Hutchinson's MT api and Browser integration from cyanogen) – Jbed – App2SD – Some IME dictionnaries – Backup/restore […]
[…] for fun i was googling around a bit today and found this nice site. It features a helper class that cleans up the multitouch event stream and should clean up the mess […]
can you post the apps with the manifest set to android 1.6 as the minSDK instead of 2.0, so that we can install in cyanogenmod phones? even though cyanongen has the support for multitouch we still cannot install because it thinks we have an older SDK (which we really do)
Uh, the fact is that these demos require the 2.0 API, so if cyanogen supports the full 2.0 API already then he needs to update the phone to let it know it is running that API. If I change the API version back to 1.6 then people will try to install it that don’t actually have the 2.0 API and the apps will crash or fail to start. Please report this as a bug in the cyanogenmod bug tracker, which is linked to from the cyanogenmod wiki, and/or ping cyanogen on IRC on #android. I think it just requires a single change in one file to let the phone know it is running a different API version.
If you’re running the older API that has my old kernel hack then you can download the same demo apps that work on the 1.1 API or later with the kernel hack, see my “Zoom-Zoom-Zoom” post on this blog.
Also, I am no longer supporting the kernel hack version because it will be gone from all phones in a few weeks or months as everyone moves to 2.0/2.1.
I have been to that zoom-zoom-zoom post, and that mtvisualizer app no longer works in CyanogenMod-4.2.13
read the notes on the 4.2.13 version and you will see. “All-new multitouch code using the 2.0 API was added. Contacts app updated. Many other bugs fixed!” keep in mind 4.2.13 is a 1.6 firmware. When reading this article I had assumed you were talking about this 4.2.13 update that cyanogenmod made. I guess you were talking about the new beta nexus cyanogen rom. But from the notes it seems either way, both roms should work with your latest MT apps because he put support in both, only problem being is that we cannot install a 2.0+ SDK app.
Am I misunderstanding something? I guess I can recompile your source and just change the manifest and give it a shot.
So I went ahead, downloaded the source, changed the manifest minsdk to 4 (1.6) and ran MTVisualizer on my HTC Dream with CyanogenMod 4.2.13 and it works.
Great, thanks for the feedback. Absolutely, it should work — and so cyanogen should change his API level to 2.0 (assuming he supports the rest of the 2.0 API — maybe he doesn’t, and thus hasn’t changed it officially?)
His Rom is a Donut ROM with some Eclair back-filling as he likes to call it. So that is why it is 1.6 still
Hi there,
thanks a lot for this very crucial piece of code/information. I was fighting for quiet some time with the multitouch api in 2.0 and also stumbled upon the inherent hardware/driver issues you discovered. I hope that i can work something out for multitouch game controls by using your code. I will of course share anything that works.
You are my new hero 🙂
Mario
hello,
I was wondering if you could help me out a bit. I’ve been trying to develop a game using multitouch and have hit a road block. I was wondering if you knew how many fingers the OnTouchEvent can register. For testing this I have been using the function event.getPointerCount().
I have been successful getting it to recognize 2, but no more than that. I thought it was suppose to recognize 3 or 4.
Thanks for your help
Great question. The current synaptics firmware only reports two touch points, so even the kernel driver doesn’t have access to more than two. And actually because it is only a 2x1D sensor not a true 2D sensor, if it reported more than two points, the chance of ambiguity would skyrocket, because for N physical touch points that are separated in X and Y, you get N^2 possible touch points that the sensor would potentially detect (I have written about this on other posts in the Android section of this blog). The Android API supports an arbitrary number of touchpoints though so the Android team is clearly thinking ahead, there just hasn’t been even a rumor or leak of an actual device with a true 2D sensor. If your game is designed to make use of more than two touch points though, when that device comes along, it should just work out of the box. However for right now, this is a hardware limitation.
[…] posted that is backwards compatible with Android 1.5 and Android 1.6, you will find it mentioned here and the direct download link is here. It is one of the features I will be implementing to […]
Fantastic – I’ve been looking for something like this. I was wondering though, is it possible to have the input points not “snap” to each other? Is it a feature for gesture recognition that I can turn off, or is it a limitation of the 2x1D hardware that you were mentioning above?
Nope, it’s a hardware limitation, sorry. There’s a lot of confusion over this so I’ll probably write another blog post explaining this sometime soon.
Thanks for the quick reply and the great API. I’m sure I’ll be able to find some neat uses for it in the future.
Great Article….
Dum question but how do you use setHandleSingleTouchEvents?
I’m trying to figure out whether this class would help w/ single touch gestures (dragging left/right or in a circular motion) or how to hook this into an app that supports both single touch and multi touch gestures…
If you don’t use handleSingleTouchEvents is false then onTouchEvent() returns false for non-multitouch events, which means the event will be passed along to the next registered touch event handler. You can register multiple touch event listeners and they will be passed the events in order until one of them returns true. You could use this to pass on single-touch events to a gesture detector, yes. Or you could combine some of the event cleanup logic from my multitouch class with logic from the gesture detector (if you have the source) to make a gesture detector that cleanly handles both single and multitouch gestures.
Hi Luke,
your MT controller seems very interesting. Is there any chance that you release the code (probably other version of the same code) with a GPL v2 license?
As you know Apace license v2 and GPL v2 are incompatible so I am not able to use your class into my code… 😦
Done! Updated the code, please feel free to go download the new version and use it in GPL-licensed projects.
Wow! Thanks for your fast update!
Hi Luke,
thanks for this great library… I’m using it in my WebcamHolmes android app, and it works like a charm!
http://code.google.com/p/webcamholmes/
More precisely, I used it in RainbowLibs project, a set of android libraries for everyday use. RaibowLibs are used by WebcamHolmes
http://code.google.com/p/rainbowlibs/
Great work, thank you!
Hi!
I would like to ask your opinion if the Android OS is capable of handling multiple mouse devices if I would install on a PC, something like MPX in Xorg.
Unfortunately in Xorg, I cannot set correctly two different mouse pointer scheme. In case I set, I end up in always changing shape.
In case Android would handle multi cursor, I would choose Android for my apps which really requires two person interaction.
I know that multitouch is more that multi-mouse… I hope some of you have an idea.
Yes, Android is capable of supporting at least one mouse, somebody hacked it to support a USB mouse on rooted systems at least — but I don’t know if multiple actual mouse cursors is supported. All recent versions of Android support multitouch however. You first need to be clear on the difference between mouse-driven and touch-driven user interfaces: interacting with a touch-driven user interface is very different (and this is why some websites break on Android or iOS — dragging something around the screen and scrolling around the page uses the same interaction method, for example, but it goes deeper than that). Also you don’t have right-click etc. Secondly if you have multiple users, you have no way of knowing which touch point belongs to which user unless you limit users to different areas of the screen, this may or may not be a problem for your UI. Thirdly there are multitouch limitations on most Android devices today (usually 2 points max; points snap together in X and/or Y, etc., as detailed elsewhere on this blog), but the devices released a year from now should all have more modern touch screen technology, so now is the time to code for true multitouch. Ultimately you should decide what platform you want to code on, and then create something that works properly on that platform, whatever the restrictions…
Hi lukeHutch,
I have a Xperia X10 (mini) which have for touch solution a synaptics clearpad2000 the same as the G1 or the nexus one.
But can’t patch and flash kernel because of Bootloader which is still closed.
So, If I succed to modify my synaptics_i2c_rmi.c ( new version from source) by inspiring me of yours.
IS there a chance to load it on kernel on the fly and override the original one ? by using and insmod or other thing?
I don’t understand very well because I’m a very young linux user.
Thanks for reading and answering and Sorry for the english.
Shok
Hi Shok,
Unfortunately you can only load kernel modules on the fly if that particular part of the kernel is compiled as a module. It’s likely that the synaptics driver is not compiled as a module, but is directly compiled into the kernel, because it’s not an optional piece of hardware. Also even if it were a module, you’d need to have root access to put the module somewhere it would be automatically loaded, or to load it manually yourself (otherwise normal users could run anything they wanted as root). So really you’re stuck until the phone is rooted. I’m surprised the bootloader is not cracked yet! Are you sure nobody has done it? If so, it’s probably only a matter of time…
Good luck,
Luke
Phone is already rooted, but bootloader will near be cracked (since months :D) but not yet.
Root is already enabled.
CUstom ROM is In progress and launch on the actual Kernel.
Sony Ericsson annonced to enable multitouch on it (2011) after someone who stay anonymous post a video showing Xperia X10 running multitouch (http://www.youtube.com/watch?v=DyI0Q4cJsDg) so this person found a way to enable it, but not explain how.
thanks for answering, I now attempt to modify it then I’ll compile the driver and try to load it.
but my synaptics_i2c_rmi.c is so different, I need to understand more what each lines does.
Hmm. Well I helped a guy get it working on an Acer Liquid, but honestly I haven’t touched this stuff in almost a couple of years. The nice thing is that you can compile and replace the kernel quite easily without having to compile the entire Android stack. Try to find a kernel configuration file for your phone first. However, the purpose of my modifications to the synaptics driver was just to hack multitouch in, and that was before there was official support for multitouch in Eclair. So actually re-compiling Android too makes more sense, because then you get the official APIs that work with all the apps, not just my hacked apps. I would ask around on XDA to see if someone else is working on this, and join forces with them — I bet you’re not the only one!
Good luck,
Luke
by searching more, I found that in Xperia X10 family there is particularity in touch solution :
X10 : Cypress truetouch Gen2 solution
X10 Mini : synaptic’s clearpad2000 solution
X10 Mini PRo : no information but it’s not clearpad 2000
SE says they will put it on X10 but nothing about X10 mini and X10 mini pro MT implementation.
So, I’ve an Mini PRo, MT seem to be far far away from me.
Hope Bootloader will be cracked Soon, sorry for disturbing you.
No problem! That’s a shame. I hate it when apparently similar devices have very different hardware, it makes life complicated for everybody. Hope you get it figured out!
Hi again ,
I finaly found all exacts particularities:
X10 : Cypress capsence controlers (DualTouch)
X10 Mini : synaptic’s clearpad2000 solution (DT)
X10 Mini PRo : CypressTrueTouch gen3 controller (MT)
so, look more like a brotherhood than like a family.
I should have real MT .
indeed, prototype driver doesn’t describe any kind of MT or DT support and I’m just an verly early developper so the way to got it work seems to be very long.
Now , I’m going to read more and more about android and linux himself, because there is to many things I don’t know about it, maybe I next found a way to activate full MT in my device.
Luke, thank you very much for the great code that you made available for the community. We just finished an app – mCatalog – that uses your multi-touch controller. It looks especially cool, when a product has many images; then the album of the images “jumps” out of the original icon and you can “pull it apart”, move around and pinch-to-zoom. Not many products in our catalog have more than one image at this time, but I hope this changes so that folks can fully enjoy the possibilities provided by your multi-touch controller.
Thanks,
A.Heiphetz
http://www.ahg.com
http://www.mobilecatalogs.net
Hi Alex, thank you for the comment. This particular blog is out of date, so I’m glad you found the up-to-date version of the code on Google Code. I’m also glad the code works well for you, thanks for letting me know it was useful. I’ll list your app on the Google Code page, it may help you get a little visibility.
ok long shot here. I know its been a few years. Now androids at 4.2 and multi touch work good. I have looked high and low for a solution/ I am trying to map the pinch to zoom function used in apps via this motion event/multi touch instruction. to a single key press.
zoom in/out keys or mapped to scroll wheel on usb hardware mouse.
Sure, see my Google Code project, android-multitouch-controller.
Hi Luke,
Thanks for nice library.. your library was great!!
I have some question, how to set image on the center of screen and how calculate center screen coordinate for image?
Hi Eko, good question, this is a bit complicated to do using the current framework. You can read the screen size as with any Android activity using getWidth() and getHeight() on the canvas element, and there are methods for converting to and from screen coordinates (e.g. when you tap on the screen, it converts backwards from screen coordinates to canvas coordinates). Take a look at the coordinate conversion code in the motion event handler.
Ya Luke, but i want to use center of screen only in first load, i’m very confuse to calculate center of screen. i’m use metrics.widthPixel/2 to get X center coordinate but wrong place not on the center screen. sorry my english is bad Luke 😀
Eko: sounds like you’re trying to center the image in canvas coordinates, not screen coordinates. The image size and position is in canvas coordinates. If the canvas has a zoom of 0.5, then widthPixel/2 will be only a quarter of the way across the screen. Start out with a canvas zoom of 1.0 and a canvas offset of (0,0), then you don’t have to translate between canvas coords and screen coords.
hi Luke, i am big friend of you, your work is awsome,currently i have use your demo code android multitouch controller this code is work for drawable image only if i want to use it for imageview object that consist bitmap how can i use it ?
Hi Nirav, thanks for your interest in using this code. It has been a long time since I have touched that Android code, but there is nothing about the code that limits it to being used with a Drawable. The multitouch controller code itself just deals with touch events, and can be used for any type of UI layer that delivers the standard events. The demo code was just a quick hack to show how to use the controller — it’s not OpenGL accelerated or anything, and I don’t recommend using it in production! Unfortunately I don’t do Android development anymore, so I can’t recommend what to do with ImageView objects.
Thanks for your quick reply,i hope you will do further android developement so we will get great demo like MT Controller and if you have extra time please try to give my question answer.. this is very useful in my project without it my work will not go further…