Window, Scene and Node coordinates in JavaFX

Facebook

Twitter

Linkedin

Here is an illustration on how the coordinate systems of the screen and JavaFX objects relate to each other. It is a great thing with JavaFX that you seldom bother about screen coordinates. Instead you rely on layout managers to position your nodes on screen. A node has its own coordinate system, should you like to do some positioning, e.g. in a game.

However, it can be difficult if you like to use a tool such as java.awt.Robot to demo or test your system. Robot has been part of Java for a long time and gives you the possibility to send stimuli to an application.

Robot works with integer screen coordinates so to click a button in my application, I need to find where it is on screen. I’ll give you the theory here and then a sample button that initiates a Robot to click on the button itself.

See the picture here. The “Window” arrow is at the point of the window’s origin in screen coordinates. Thus, if the user moves the window on the screen, the origin changes. The “Scene” arrow is at the point of the scene’s origin in window coordinates, in my case (8, 30). The node, which may be nested deep within a scene graph, can get its point in screen coordinates by using the method localToScene.

By that, we know enough to find the position of our button so that Robot can click on it. We simply add the window, scene and node coordinates.

As said, here is the code for the button that clicks itself. The buttonActionHandler returns an event handler that should be used for the button.

        final Button btn = new Button();
        btn.setText("Say 'Hello World'");
        btn.setOnAction(buttonActionHandler(btn));

The handler prints out a famous message and calculates where the button is on screen. Note that we use the button’s upper left corner, which isn’t necessarily where we always wish to go. The coordinates are sent to clicMeAgainLater which sets up a TimeLine that will 5 seconds later call clickIt. That is where we use a Robot to move the mouse to the given point, press and release the left mouse button. Detailed stuff, as you can see. 🙂

 private EventHandler<ActionEvent> buttonActionHandler(final Button button) {
        return new EventHandler<ActionEvent>() {
            @Override
            public void handle(ActionEvent event) {
                System.out.println("Hello World!");
                final Scene scene = button.getScene();
                final Point2D windowCoord = new Point2D(scene.getWindow().getX(), scene.getWindow().getY());
                final Point2D sceneCoord = new Point2D(scene.getX(), scene.getY());
                final Point2D nodeCoord = button.localToScene(0.0, 0.0);
                final double clickX = Math.round(windowCoord.getX() + sceneCoord.getX() + nodeCoord.getX());
                final double clickY = Math.round(windowCoord.getY() + sceneCoord.getY() + nodeCoord.getY());
                clickMeAgainLater(clickX, clickY);
            }

            private void clickMeAgainLater(double x, double y) {
                TimelineBuilder
                        .create()
                        .keyFrames(
                           new KeyFrame(Duration.seconds(5), clickIt(x, y)))
                        .build()
                        .play();
            }

            private EventHandler<ActionEvent> clickIt(final double x, final double y) {
                return new EventHandler<ActionEvent>() {
                    @Override
                    public void handle(ActionEvent t) {
                        try {
                            Robot robot = new Robot();
                            robot.mouseMove(new Double(x).intValue(), new Double(y).intValue());
                            robot.mousePress(InputEvent.BUTTON1_MASK);
                            robot.mouseRelease(InputEvent.BUTTON1_MASK);
                        } catch (AWTException ex) {
                            logger.log(Level.SEVERE, "bad mouse", ex);
                        }
                    }
                };
            }
        };
    }