Using multi-touch and sensors in Java Felipe Pedroso
Agenda •The future of PCs •JavaFX and Touch Support •Using JNI to work with sensors 2
Mobile Computing Lines are Blurring Ultrabook ™ Ultrabook™ Notebook Smartphone Tablet Convertible & Detachable Consumption Usages Creation/Productivity
Reshaping the PC Experience Multi-Touch Ambient Multi-Touch Ambient Acceleromet Light Acceleromet Light er Sensor er Sensor Ultrabook™ Compass GPS Compass GPS Near Field Near Field Gyroscope Communicatio Gyroscope Communicatio n n Context Aware Sensors 4
OK, they have all those features… … but how can we implement them using Java?
To allow users to TOUCH my Java App... • Should I use... • AWT? • Swing? • SWT? • Actually, you can use them but... • They aren’t made for multi-touch (OK, there’s a way to do it) • You must optimize your UI controls to be more ‘touch friendly’ • So, how to do it in a simple and easy way? 6
Actions supported • Touch events: Down, Move and Up • Tap (Down and Up) / Double tap • Drag and Drop (Down, Move and Up) • Hold • Gestures • Swipe: Up, Down, Left and Right • Zoom: Pinch / Spread • Rotate 8
What I need to do to manipulate components? • Your components must extend the Node class or any of its subclasses (StackPane, ImageView, etc) • Set the proper EventHandler to handle the action.
What I need to do to manipulate components? • Apply a transform to the component (Translation, Rotation and Scale): • Let’s dive into some code!
What about the sensors? JNI, I choose you!
Available sensors on Ultrabooks • Accelerometer • Proper acceleration in three axis (x, y, z) • Gyrometer • Device orientation • Magnetometer • Strenght and direction of magnetic fields • GPS (Global Positioning System) • Location and Time information • NFC • Near Field Communication • Ambient Light Sensor • Ambient Light Level 12
Windows API – How to • Get the default object of your sensor using the GetDefault method • You can call the GetCurrentReading() to get the current value of the sensors or... • ... work with the ReadingChanged event • Set the attribute ReportInterval (please, respect the MinimumReportInterval to avoid problems) • Delegate a method to handle the event (something like setting a method as a listener) • Handle the event! • This procedure is valid only for sensors from Windows.Sensors namespace 15
How can I access that? 16
Using JNI to access the sensors from Windows.Sensors 1. [Java] Create a native method to register the object that will handle the events that come from the sensor 2. Generate the header file using javah 3. [C++] Create a DLL Project in Visual Studio with the DLLs and namespaces of the Windows 8 APIS 4. [C++] Use a variable to keep a reference to the object and the ID (GetMethodID) of the method that will handle the event 17
Using JNI to access the sensors from Windows.Sensors 5. [C++] Initialize the sensor and delegate a C++ method to handle the event 6. [C++] Redirect the delegate method call to the Java method using the function CallVoidMethod with the following parameters: the Java Object, the method ID and it’s parameters 7. [Java] Handle the event! A lot of text? Please, show me the code! 18
What about Linux? • Touch works fine, but there aren’t clear APIs to read sensors. If you know how to do it, let’s work together! 19