• Download Eclipse Project - 9.8 KB

Part 2a - VTK Basics

Now we're cooking with gas and starting the 3D user interface... This article also contains the first upload of the completed project code. Here is a quick teaser screenshot of the 3D VTK UI at the end of Part 2c:

The fastest way to learn VTK is to watch the KitWare VTK Overview movie. This will save you days of debugging (sigh... like I did... overconfidence always gets ya).

If you have more spare time and really want to start digging into VTK, you will need to get familiar with the VTK datastructures. You can read up on the datastructures in following articles (and you could also purchase the  books if you want even more detail):

• Data Structures in the Visualization Toolkit - Stefan Bruckner

• Visualizing with VTK: A Tutorial

• VTK Tutorial

So from this point we are assuming that you are familiar with readers, mappers, actors, etc. You needn't be able to build out a VTK application right away, but should be familiar with the concepts. Effectively that means that you understand what the program in the previous article is doing. We'll build on this to set up our 3D environment.

This will be done in the following way:

• We'll create an Eclipse PyDev project at this point. This will become the backbone of the complete system

• Two classes will be built:

  • A main class for the program entry point

  • A SceneObject class that is the beginning of a parent class for objects in the 3D world

• This article will conclude by adding a sphere object back into the world using the SceneObject template

• The next article (part 2b) will work through a few VTK scenarios by basically saying 'I want to draw spheres/create a 3rd-person camera/draw complex models...' and discussing how to do that, almost in the form of independent topics around doing specific things in VTK

• Part 2c will sum up by building out a 3D environment that is our representation of the world from the bots' perspectives

• The remaining articles will use this project to add in the 2D PyQt and LCM components

Finally, this works through VTK in an 'I'm building a game' way. This is because, unlike general VTK users working with scientific graphing, we'd argue that we're actually building a real-time simulation environment, ~ a game. Back in the day (...quickly putting in my dentures...) when we taught game design, we would introduce a toolbox of neat things you could use - e.g. skinned models, deforming terrains, pixel shaders - and the students chose to use what they wanted for their game. This post follows a similar train of thought.

Creating a PyDev Project

To create our base project, you will need to:

1. In Eclipse (with the PyDev perspective set) click 'File' -> 'New' -> 'PyDev Project':

   1. Give your project a name, I used 'bot_vis_platform'

   2. Click 'Finish' and skip to the 3rd point

   3. If you can't click 'Finish' because it's greyed out as shown below, you probably need to configure an interpreter:

      

   4. Click 'Please configure an interpreter before proceeding'

   5. Click 'Quick Auto-Config'

   6. You should then be able to click 'Finish' and create your project

2. Now that the project is created, you will need to create a main file and a few package folders for the future components:

   1. In the 'PyDev Package Explorer', right-click on the root node 'bot_vis_platform' and select 'New' -> 'PyDev Package' and give it the same name - 'bot_vis_platform' (this will be our root package that will reference the VTK, LCM, and PyQt components):

      

   2. On the same root node 'bot_vis_platform' in the package explorer, select 'New' -> 'PyDev Package' and create a package called 'scene' (we'll build all the VTK components in this folder)

   3. Lastly, in the 'bot_vis_package', right-click again, select 'New' -> 'PyDev Module', and create a module called 'bot_vis_main'. This will be our main program, so when it asks for a template, select 'Module: main' (this just gives you a bit of boilerplate code):

      

   4. Your final project setup should resemble the screenshot below and you should be able to run the program (the next section will start building out something that you can interact with):

      

Building a Main Rendering Loop

For the moment, we're going to use a 'vtkRenderWindowInteractor' to handle the rendering. This will take care of the main rendering loop as well as interacting with the world. For a placeholder, we'll put down a sphere source and let the camera automatically focus on that. Replace the 'bot_vis_main' program with the code below.

You should see the familiar sphere if you run the module by clicking on the down arrow on the right of the play button on the top toolbar, and select 'bot_vis_platform bot_vis_main'):

In some cases that option isn't available, if you can't do that, just right-click on 'bot_vis_main.py' in the package/project explorer and use the Run menu there.

import vtk

if __name__ == '__main__':

    # Sphere
    sphereSource = vtk.vtkSphereSource()
    sphereSource.SetCenter(0.0, 0.0, 0.0)
    sphereSource.SetRadius(4.0)

    sphereMapper = vtk.vtkPolyDataMapper()
    sphereMapper.SetInputConnection(sphereSource.GetOutputPort())

    sphereActor = vtk.vtkActor()
    sphereActor.SetMapper(sphereMapper)
    # Change it to a red sphere
    sphereActor.GetProperty().SetColor(1.0, 0.0, 0.0);

    # A renderer and render window
    renderer = vtk.vtkRenderer()
    renderWindow = vtk.vtkRenderWindow()
    renderWindow.AddRenderer(renderer)
    # Make it a little bigger than default
    renderWindow.SetSize(800, 600)

    # An interactor
    renderWindowInteractor = vtk.vtkRenderWindowInteractor()
    renderWindowInteractor.SetRenderWindow(renderWindow)

    # Add the actors to the scene
    renderer.AddActor(sphereActor)

    # Render an image (lights and cameras are created automatically)
    renderWindow.Render()

    # Begin mouse interaction
    renderWindowInteractor.Start()
    renderWindowInteractor.Initialize()

    pass

A quick overview of the code:

• As the video discussed, this creates a 'vtkSphereSource', which generates the actual mesh data structure

• The data is passed to a mapper, which interprets the data in the data structure (here we're connecting to a pretty standard source so we can use a 'vtkPolyDataMapper' which knows how to interpret the sphere mesh data)

• The mapper is in turn bound to an actor, which encapsulates things like the texture, position, and orientation of the model. The three working together draw the sphere (source -> mapper -> actor).

• We then create a renderer and a render window for seeing the scene.

• A 'vtkRenderWindowInteractor' is created and set to control the render window. This allows you to move around in the scene without having to write any code and also handles the rendering loop

• The actor for the sphere is then added to the renderer, and the 'vtkRenderWindowInteractor' is run (with the '.Start()' and '.Initialize()' calls) to block the main program in an interactive rendering loop

This is standard code for the main loop for this project, so with the exception of the sphere itself (which will be removed in the next section), not much will change here until we add in custom cameras.

A Base 'SceneObject' Class

From this point we could potentially just continue adding in code to the main loop, as was done with the sphere. The only problem is that grows really quickly into something unmanageable. To fix this, I'd like to introduce the start of a parent class for any object that will be the 3D scene. This won't contain too much, it's just the start of a template, but it should:

1. Have a standard actor that encapsulates the position and orientation of any object in the 3D world

2. Control any children that are attached to it, i.e. bound by position and orientation, so that if we move the parent the children move

3. In some cases we want to offset the children from the origin of the parent (by position or rotation), so it should automagically manage that

4. For the moment we are just going to be moving these around or changing their orientations - it should also have standard methods for that

Adding a SceneObject to the Scene Module

The 'SceneObject' class is going to exist in the 'scene' package, to add it in:

1. Right-click on the 'scene' package and select 'New' -> 'PyDev Module'

2. Give it the name 'SceneObject'

3. When it asks which template you want, just select 'Module: Class'

4. Rename the highlighted class name to 'SceneObject'

Your code for the 'SceneObject' should look like the screenshot below:

Common SceneObject Fields and Initialization

We want to have common fields for any object that derives from 'SceneObject' (i.e. is a child of it) and exists in the scene.

For the moment, these are just a common actor as well as possible children fields. The children fields allow us to build a tree of 'SceneObjects' so that you can build a compound class.

A good example of this is provided in the next article, where we have a Bot class that has visualized sensor data as children (a camera screen and a LIDAR point cloud) which move around with it. If we structure the 'SceneObject' class just right, this requires almost no effort to do.

One thing to note: We would like to be able to offset the children, either by position or by rotation, from the parent. If we don't, all the children will be drawn at the center of the parent (which sucks a bit), so additional fields are included to allow you to do this... This isn't a full forward kinematic system, just enough to get started. These are the 'childX' members.

Also, we want our actor to be added to the scene when we construct it, so that we don't need to worry about that later (it will automatically draw it if the actor is wired up to a valid mapper). The renderer is therefore passed into the 'SceneObject' constructor and when the actor is instantiated, it is added to the scene right away. The code snippet for the constructor of 'SceneObject' is:

def __init__(self, renderer):
    '''
    Constructor with the renderer passed in
    '''
    # Initialize all the variables so that they're unique to self
    self.childrenObjects = []
    self.childPositionOffset = [0, 0, 0]
    self.childRotationalOffset = [0, 0, 0]
    self.vtkActor = vtk.vtkActor()
    renderer.AddActor(self.vtkActor)

Adding in Getters and Setters

The last thing to add in is the getters and setters for the 'SceneObject's position and orientation. This allows us to move the whole 'SceneObject' without worrying about the children, and these methods should be used instead of talking directly to the 'vtkActor' field. I'm not going to go into too much detail about these methods, they should be relatively straightforward, but feel free to comment if you have any questions about them. A few small tips:

• If you call a get method, you should receive a list of 3 points (either XYZ location or rotation depending on the getter), and if you use the respective setter you just need to pass in a list of 3 points, e.g. 'myObject.SetOrientationVec3([90,180,270])'

• All orientations are in degrees

The code snippet for the getters and setters of 'SceneObject' is:

def SetPositionVec3(self, positionVec3):
    self.vtkActor.SetPosition(positionVec3[0], positionVec3[1], positionVec3[2])
    # Update all the children
    for sceneObject in self.childrenObjects:
        newLoc = [0, 0, 0]
        newLoc[0] = positionVec3[0] + sceneObject.childPositionOffset[0]
        newLoc[1] = positionVec3[1] + sceneObject.childPositionOffset[1]
        newLoc[2] = positionVec3[2] + sceneObject.childPositionOffset[2]
        sceneObject.SetPositionVec3(newLoc)

def GetPositionVec3(self):
    return self.vtkActor.GetPosition

def SetOrientationVec3(self, orientationVec3):
    self.vtkActor.SetOrientation(orientationVec3[0], orientationVec3[1], orientationVec3[2])
    # Update all the children
    for sceneObject in self.childrenObjects:
        newOr = [0, 0, 0]
        newOr[0] = orientationVec3[0] + sceneObject.childRotationalOffset[0]
        newOr[1] = orientationVec3[1] + sceneObject.childRotationalOffset[1]
        newOr[2] = orientationVec3[2] + sceneObject.childRotationalOffset[2]
        sceneObject.SetOrientationVec3(newOr)

def GetOrientationVec3(self):
    return self.vtkActor.GetPosition()

Great! That was all the grungy work, the next part is the cool bit. With that sorted out, this class will be used to build a few simple objects for the scene.

Creating Simple Models

VTK has a huge number of different primitives you can start with, so you don't need to jump to loading complex models straight away. This section will introduce a few common objects, but there is far more documentation and examples at Geometric Objects in vtk/Examples/Python. First of all, you should remove the 'hacked in' sphere that we had in the main loop... Makes my hair stand on end thinking that we have that code in a main loop. Your main loop should look something like the following:

if __name__ == '__main__':

    # A renderer and render window
    renderer = vtk.vtkRenderer()
    renderWindow = vtk.vtkRenderWindow()
    renderWindow.AddRenderer(renderer)
    # Make it a little bigger than default
    renderWindow.SetSize(1024, 768)

    # An interactor
    renderWindowInteractor = vtk.vtkRenderWindowInteractor()
    renderWindowInteractor.SetRenderWindow(renderWindow)

    # [INSERT COOL STUFF HERE]

    # Render an image (lights and cameras are created automatically)
    renderWindow.Render()

    # Begin mouse interaction
    renderWindowInteractor.Start()
    renderWindowInteractor.Initialize()

    pass

We will now build a few primitives using the 'SceneObject' class. Specifically we will reintroduce the sphere, add in a set of cylinders, and draw an axes gidget. In these sections, I assume that you are comfortable adding in new classes and files. Just right-click on the 'scene' folder in the package explorer and select 'New' -> 'PyDev Module'.

Adding in a Sphere Primitive

The first is a simple sphere, exactly the same as we had it in the earlier sections. The complete code for the 'sphere.py' file is:

import vtk
from SceneObject import SceneObject

class Sphere(SceneObject):
    '''
    A template for drawing a sphere.
    '''

    def __init__(self, renderer):
        '''
        Initialize the sphere.
        '''
        # Call the parent constructor
        super(Sphere,self).__init__(renderer)

        sphereSource = vtk.vtkSphereSource()
        sphereSource.SetCenter(0.0, 0.0, 0.0)
        sphereSource.SetRadius(4.0)
        # Make it a little more defined
        sphereSource.SetThetaResolution(24)
        sphereSource.SetPhiResolution(24)

        sphereMapper = vtk.vtkPolyDataMapper()
        sphereMapper.SetInputConnection(sphereSource.GetOutputPort())

        self.vtkActor.SetMapper(sphereMapper)
        # Change it to a red sphere
        self.vtkActor.GetProperty().SetColor(1.0, 0.0, 0.0);

This code will create a red sphere with a radius of 4 at the origin. Let's quickly discuss the components here:

from SceneObject import SceneObject

• This line imports the 'SceneObject', which we will inherit from

class Sphere(SceneObject):

• This defines a 'Sphere' class that inherits 'SceneObject', which means that it inherits all the fields and methods of the parent class (like our 'vtkActor')

def __init__(self, renderer):
    '''
    Initialize the sphere.
    '''
    # Call the parent constructor
    super(Sphere,self).__init__(renderer)

• The constructor '__init__(self, renderer)' will be passed the vtk renderer when it is created (self is a special parameter, it is ignored)

• When created it will call the parent constructor with the renderer, which will add the object to the renderer (how awesome is inheritance?) - this is done in the 'super(Sphere, self).__init__(renderer)' line and is read as "call the superclass constructor of Sphere with me (self) and give it a renderer"

sphereSource = vtk.vtkSphereSource()
sphereSource.SetCenter(0.0, 0.0, 0.0)
sphereSource.SetRadius(4.0)
# Make it a little more defined
sphereSource.SetThetaResolution(24)
sphereSource.SetPhiResolution(24)

• Create a local 'vtkSphereSource' exactly as we did in the main function and set it be at the origin with a radius of 4

• Set the resolution of the mesh to slightly higher than normal so that it looks like a sphere and not a prop from a 1980's music video

sphereMapper = vtk.vtkPolyDataMapper()
sphereMapper.SetInputConnection(sphereSource.GetOutputPort())

• Create a mapper exactly as we did in the previous example and assign the 'sphereSource' to it as an input

self.vtkActor.SetMapper(sphereMapper)

• The actor was already initialized in the parent class, so set the actor's mapper to the sphere

• Note: It's really important to call 'self.vtkActor', so that it's assigned to the instance (self) of 'vtkActor'. If you skip that it will assign it to the shared class field, which could cause a potential nightmare in the future

# Change it to a red sphere
self.vtkActor.GetProperty().SetColor(1.0, 0.0, 0.0)

• Lastly, set the sphere to be red by telling the actor to use red (RGB = (1, 0, 0)). There are quite a few neat 'vtkActor' properties you can set, more can be found at the 'vtkActor' wiki

To draw this in the main scene, we just need to add a few lines to the main program 'bot_vis_main.py'. I'll add in a large piece of the main loop code here, but in the next sections we'll just work with the snippets (this post is way too long already and I could eat the legs off a low flying duck). The main loop will then look like:

import vtk

from scene import Sphere

if __name__ == '__main__':
...
    # [INSERT COOL STUFF HERE]

    # Add in two spheres
    sphere1 = Sphere.Sphere(renderer)
    sphere1.SetPositionVec3([0, 6, 0])
    sphere2 = Sphere.Sphere(renderer)
    sphere2.SetPositionVec3([0, -6, 0])
...

A few points on what was added:

• The 'Sphere' class was imported from the 'scene' package

• Two spheres were created, both being passed the renderer

• Each sphere was moved using the new setters - one at +6 on the Y axis, and the other at -6 on the Y axis. Note that the XYZ values were passed in as lists

When you run this code, you should the two spheres that we declared. Pretty neat, right?

Adding in a Cylinder Primitive

Working from the 'Sphere' example, the code for a cylinder is provided below and in the project as 'Cylinder.py'.

import vtk
from SceneObject import SceneObject

class Cylinder(SceneObject):
    '''
    A template for drawing a cylinder.
    '''

    def __init__(self, renderer):
        '''
        Initialize the cylinder.
        '''
        # Call the parent constructor
        super(Cylinder,self).__init__(renderer)

        cylinderSource = vtk.vtkCylinderSource()
        cylinderSource.SetCenter(0.0, 0.0, 0.0)
        cylinderSource.SetRadius(2.0)
        cylinderSource.SetHeight(8.0)
        # Make it a little more defined
        cylinderSource.SetResolution(24)

        cylinderMapper = vtk.vtkPolyDataMapper()
        cylinderMapper.SetInputConnection(cylinderSource.GetOutputPort())

        self.vtkActor.SetMapper(cylinderMapper)
        # Change it to a red sphere
        self.vtkActor.GetProperty().SetColor(0.8, 0.8, 0.3);

To use this in the main class, you would use the same code as the 'Sphere' class. To make it slightly more interesting, let's create a few and space them around the primary axis of the spheres (around the XZ plane) using a circle formula:

import vtk
from math import sin,cos

from scene import Cylinder
from scene import Sphere

if __name__ == '__main__':
...
    # [INSERT COOL STUFF HERE]
...
    # Add in 8 cylinders
    numCyls = 8
    for i in xrange(0,numCyls):
        cylinder = Cylinder.Cylinder(renderer)
        # Note that although VTK uses degrees, Python's math library uses radians, so these offsets are calculated in radians
        position = [10.0 * cos(float(i) / float(numCyls) * 3.141 * 2.0), 0, 10.0 * sin(float(i) / float(numCyls) * 3.141 * 2.0)]
        cylinder.SetPositionVec3(position)

A few points on this code snippet:

• Don't forget to import 'Cylinder' in the main class

• We use a loop to create 8 individual cylinders

• Each cylinder is spaced around the spheres using the circle formula x = cos(angle), z = sin(angle)

When you run this code, you should see the following image:

An Axes Gidget

Lastly, we are going to hack the structure a bit and introduce a useful visualization component - a set of axes. This is a slight deviation from the classes we are working with (we need a special actor to use the gidget, a 'vtkAxesActor'), so we'll run through the code in a bit of detail. The complete code for the 'Axes.py' class is:

import vtk
from SceneObject import SceneObject

class Axes(SceneObject):
    '''
    A template for drawing axes.
    Shouldn't really be in a class of it's own, but it's cleaner here and like this we can move it easily.
    Ref: http://vtk.org/gitweb?p=VTK.git;a=blob;f=Examples/GUI/Tcl/ProbeWithSplineWidget.tcl
    '''

    def __init__(self, renderer):
        '''
        Initialize the axes - not the parent version, we're going to assign a vtkAxesActor to it and add it ourselves.
        '''
        # Skip the parent constructor
        #super(Axes,self).__init__(renderer)

        # Ref: http://vtk.org/gitweb?p=VTK.git;a=blob;f=Examples/GUI/Tcl/ProbeWithSplineWidget.tcl
        self.vtkActor = vtk.vtkAxesActor()
        self.vtkActor.SetShaftTypeToCylinder()
        self.vtkActor.SetCylinderRadius(0.05)
        self.vtkActor.SetTotalLength(2.5, 2.5, 2.5)
        # Change the font size to something reasonable
        # Ref: http://vtk.1045678.n5.nabble.com/VtkAxesActor-Problem-td4311250.html
        self.vtkActor.GetXAxisCaptionActor2D().GetTextActor().SetTextScaleMode(vtk.vtkTextActor.TEXT_SCALE_MODE_NONE)
        self.vtkActor.GetXAxisCaptionActor2D().GetTextActor().GetTextProperty().SetFontSize(25);
        self.vtkActor.GetYAxisCaptionActor2D().GetTextActor().SetTextScaleMode(vtk.vtkTextActor.TEXT_SCALE_MODE_NONE)
        self.vtkActor.GetYAxisCaptionActor2D().GetTextActor().GetTextProperty().SetFontSize(25);
        self.vtkActor.GetZAxisCaptionActor2D().GetTextActor().SetTextScaleMode(vtk.vtkTextActor.TEXT_SCALE_MODE_NONE)
        self.vtkActor.GetZAxisCaptionActor2D().GetTextActor().GetTextProperty().SetFontSize(25);         

        # Add the actor.
        renderer.AddActor(self.vtkActor)

A few points on this code:

# Skip the parent constructor
#super(Axes,self).__init__(renderer)

• We need to use the 'vtkAxesActor' VTK class to include the axes gidget, so we have to skip the superclass initializer

# Ref: http://vtk.org/gitweb?p=VTK.git;a=blob;f=Examples/GUI/Tcl/ProbeWithSplineWidget.tcl
self.vtkActor = vtk.vtkAxesActor()
self.vtkActor.SetShaftTypeToCylinder()
self.vtkActor.SetCylinderRadius(0.05)
self.vtkActor.SetTotalLength(2.5, 2.5, 2.5)

• Instead, create the special actor and change a few properties that make it more visible

        # Change the font size to something reasonable
        # Ref: http://vtk.1045678.n5.nabble.com/VtkAxesActor-Problem-td4311250.html
        self.vtkActor.GetXAxisCaptionActor2D().GetTextActor().SetTextScaleMode(vtk.vtkTextActor.TEXT_SCALE_MODE_NONE)
        self.vtkActor.GetXAxisCaptionActor2D().GetTextActor().GetTextProperty().SetFontSize(25);
...

• Set the text on each axis to slightly smaller than default - this is a copy from the reference link, I'd recommend just reading up on it for interest's sake

# Add the actor.
renderer.AddActor(self.vtkActor)

• Unlike the normal code, we now need to add the actor to the renderer because we skipped the parent constructor, which did the addition for us

To use this code in the main function, nothing changes. Here is a quick snippet of the changes and an image of the result:

from scene import Axes
from scene import Cylinder
from scene import Sphere

if __name__ == '__main__':
...
    # [INSERT COOL STUFF HERE]
...
    # Add in a set of axes
    axes = Axes.Axes(renderer)

Next Article

Done! You can download the complete project at the top of this article. The next article will introduce some slightly more complex topics, such as:

• Complex models

• Terrains

• Camera images using textures

• A LIDAR representation using point clouds

<mytubeelement data="{"bundle":{"label_delimitor":":","percentage":"%","smart_buffer":"Smart Buffer","start_playing_when_buffered":"Start playing when buffered","sound":"Sound","desktop_notification":"Desktop Notification","continuation_on_next_line":"-","loop":"Loop","only_notify":"Only Notify","estimated_time":"Estimated Time","global_preferences":"Global Preferences","no_notification_supported_on_your_browser":"No notification style supported on your browser version","video_buffered":"Video Buffered","buffered":"Buffered","hyphen":"-","buffered_message":"The video has been buffered as requested and is ready to play.","not_supported":"Not Supported","on":"On","off":"Off","click_to_enable_for_this_site":"Click to enable for this site","desktop_notification_denied":"You have denied permission for desktop notification for this site","notification_status_delimitor":";","error":"Error","adblock_interferance_message":"Adblock (or similar extension) is known to interfere with SmartVideo. Please add this url to adblock whitelist.","calculating":"Calculating","waiting":"Waiting","will_start_buffering_when_initialized":"Will start buffering when initialized","will_start_playing_when_initialized":"Will start playing when initialized","completed":"Completed","buffering_stalled":"Buffering is stalled. Will stop.","stopped":"Stopped","hr":"Hr","min":"Min","sec":"Sec","any_moment":"Any Moment","popup_donate_to":"Donate to","extension_id":null},"prefs":{"desktopNotification":true,"soundNotification":true,"logLevel":0,"enable":true,"loop":false,"hidePopup":false,"autoPlay":false,"autoBuffer":false,"autoPlayOnBuffer":false,"autoPlayOnBufferPercentage":42,"autoPlayOnSmartBuffer":true,"quality":"default","fshd":false,"onlyNotification":false,"enableFullScreen":true,"saveBandwidth":false,"hideAnnotations":false,"turnOffPagedBuffering":false}}" event="preferencesUpdated" id="myTubeRelayElementToPage">