Arduino and Processing code error "disabling serialEvent()"

Question

I am facing the following error in Processing.

I'm using Windows 10, Processing version 3.5.4, and Arduino version 1.8.16.

Error, disabling serialEvent() for COM3 null

Processing code:

import processing.serial.*;
import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.InputEvent;
import java.awt.event.KeyEvent;
import javax.swing.KeyStroke;
Serial MyPort;
String KeyString = "";

void setup()
{
  System.out.println("HI");
  size(700, 500);
  MyPort = new Serial(this, "COM3", 9600);
  MyPort.bufferUntil('\n');
}

void draw() {
  background(0, 0, 0);
  fill(255, 0, 0);
  text("Press any key", 100, 175);
}

void serialEvent(Serial MyPort)throws Exception {
   KeyString = MyPort.readStringUntil('\n');
   KeyString = KeyString.substring(0, KeyString.indexOf(':'));
   System.out.println(KeyString);
   Robot Arduino = new Robot();

   switch(KeyString) {

     case "Up":
       Arduino.keyPress(KeyEvent.VK_UP);
       Arduino.keyRelease(KeyEvent.VK_UP);
       break;

     case "Down":
       Arduino.keyPress(KeyEvent.VK_DOWN);
       Arduino.keyRelease(KeyEvent.VK_DOWN);
       break;

     case "Left":
       Arduino.keyPress(KeyEvent.VK_LEFT);
       Arduino.keyRelease(KeyEvent.VK_LEFT);
       break;

     case "Right":
       Arduino.keyPress(KeyEvent.VK_RIGHT);
       Arduino.keyRelease(KeyEvent.VK_RIGHT);
       break;
   }

}

Or is there a way to control the buttons (up, down, left, right, any...)?

Answer 1

That error occurs whenever an exception is thrown within the method.

You've chosen not to handle exceptions when creating the Robot instance.

I recommend initialising Robot once in setup rather than over and over again (whenever there's a new serial message). Additionally, it might help to use Java naming conventions such as reserving TitleCase for class names and camelCase for instance names.

The main error is probably with this line though:

KeyString = KeyString.substring(0, KeyString.indexOf(':'));

This is optimistically assuming there are absolutely no errors with serial communication and the always is a : character. If something goes wrong and the string either split or empty, indexOf(':') will return -1 (not found) which makes .substring(0, -1) and illegal character index range.

Here's a modified version of your code with above suggestions applied:

import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.InputEvent;
import java.awt.event.KeyEvent;
import javax.swing.KeyStroke;

import processing.serial.*;

// Arduino serial port
Serial arduino;
// input string from Arduino
String keyString = "";
// keyboard shortcut trigger
Robot keyboard;

void setup()
{
  println("HI");
  size(700, 500);

  try{
    arduino = new Serial(this, "COM3", 9600);
    arduino.bufferUntil('\n');
  }catch(Exception e){
    println("error setting up Arduino: double check the port name is correct, present in Device Manager and not used by other apps (e.g. Serial Monitor)");
    e.printStackTrace();
    exit();
  }

  try{
   keyboard = new Robot();
  }catch(Exception e){
    println("Error setting up Robot keyboard trigger");
    exit();
  }

}
void draw(){
  background(0, 0, 0);
  fill(255, 0, 0);
  text("Press any key", 100, 175);
}

void serialEvent(Serial arduino) {
  keyString = arduino.readString('\n');
  // handle edge case
  if(keyString == null){
    // optional: println("null string: skipping")
    return;
  }
  // remove any whitespace
  keyString.trim();
  if(keyString.isEmpty()){
    // optional: println("read empty string: skipping")
    return;
  }

  // extract 1st section until ':'
  int separatorIndex = keyString.indexOf(':');
  if(separatorIndex < 0){
    // optional: println("no ':' found: skipping")
    return;
  }

  keyString = keyString.substring(0, separatorIndex);
  println(keyString);

  switch(keyString){
    case "Up" :
      keyboard.keyPress(KeyEvent.VK_UP);
      keyboard.keyRelease(KeyEvent.VK_UP);
      break;
    case "Down" :
      keyboard.keyPress(KeyEvent.VK_DOWN);
      keyboard.keyRelease(KeyEvent.VK_DOWN);
      break;
    case "Left" :
      keyboard.keyPress(KeyEvent.VK_LEFT);
      keyboard.keyRelease(KeyEvent.VK_LEFT);
      break;
    case "Right" :
      keyboard.keyPress(KeyEvent.VK_RIGHT);
      keyboard.keyRelease(KeyEvent.VK_RIGHT);
      break;
  }

}

I would also double check test the keyPress()/keyRelease() calls in isolation to ensure that the above code will work as intended (and won't require a delay between press and release).

If you simply to need to send a keypress you can send a single byte(value from 0-255). The actual keyCode for the arrow keys are 37(left), 38(up), 39(right), 40(down) so you could do send this from Arduino:

Serial.write(38);

and on the Processing side read a single byte and check it's value. Remember that in Java a byte is from -127 to 128 instead of 0-255, so you'd need to mask it, for example:

if(arduino.read() & 0xFF == 38) println("up");

Also note that there are other microcontrollers that could work as HID device (e.g., USB keyboard or mouse) instead of serial port. Makey Makey is a cute example of that, but there are also Arduino boards using the ATmega32U4 chip which can act as HID device (e.g., Arduino Leonardo, 32U4 breakouts, etc.). For this, check out the Keyboard Arduino Reference.

The advantage of this option would be simplicity: one less failure point (the serial communication software to trigger the keypresses). This disadvantage would be sending additional data. For example, if you need to send some extra values/data after the ':' that won't be straightforward to replace with keypresses.