Hi, my name is Daniel. This is a blog of my 4th year software engineering project at The University of Sydney (ELEC4707).

The title of my project is Painting with the Wii. In short, this project involves integrating DirectX into a program designed for painting with a Wii remote (or Wiimote).

I will be continually updating this blog throughout the course of the project. Please check back reguarly for updates!

Friday, October 17, 2008

Usability Study and Research Paper

Much has happened in the last 2 weeks.

- I have finished coding, completeing all the minimum requirements of the application, and some of the extended requirements. There is unlikely to be any further time to code later, due to the fact that I have to complete the thesis.

- I conducted a usability study for WiiPaintPlus, recuiting 20 participants to test the application and complete a survey about its usefulness. This took some time (between 15 and 20 hours), but the feedback has been very useful.

Here are some creations that testers made during usability testing:

- Right now, me and my supervisor (Irena) are writing a research paper about WiiPaintPlus, and the usability study that was undertaken, to be submitted to a computing journal. The main focus of the paper is on describing the creation of WiiPaintPlus, and its use of the Wiimote as an input. The Wiimote has provided loads of functionality due to its various input capabilities... and there is a lot of potential for application development using the Wiimote.

The paper is due for submission later today...

Tuesday, September 30, 2008

1 month to go...

Here's a video with some new functionality, including the tool selector and undo / redo.

Thursday, September 25, 2008

Over this past week I have been working on a number of aspects of the program.

1. I have modified the current circle and rectangle tools to make them more useful. Instead of drawing a continuous line of circles or rectangles, each click of the Wiimote's A button draws a single circle / rectangle, which increases in size the longer the A button is pressed.

Originally, I had another plan, of creating a click and drag tool for rectangles and circles, where the user can exactly specify the size of the shape by clicking and dragging. However, this had proved to place too much load on the program, especially when it is a large drawing (lots of shapes), as the whole pane needs to be redrawn each frame (to allow for increasing and decreasing rectangle size).

2. I have also created a save form, to handle saving images. The images are saved to a default location (C:\WPP) and the pane can be saved as a bitmap or jpeg. I may possiblly introduce my own file type (.wpp) if I manage to get shape selection working - a part of the extended functionality. To save an image, the user presses the down arrow on the wiimote (up will be used to Load, if I get this working).

3. I have modified the clear form, to display larger buttons and text. To clear the canvas, the user presses the Home button of the Wiimote

4. I have added a close button to the form.

5. I have implemented Undo / Redo for the most recent change to the drawing canvas. Pressing Left on the Wiimote performs an Undo, while pressing Right does a redo of what was just undone.

And that's about it...

The biggest challenges I have left to overcome are:
- Making the Shape selection tool
- Making the colour selection cube work...

I will try and upload some videos of what I have described in this post asap.

Thursday, September 18, 2008

Completed Line Tool

I managed to figure out a better way to draw lines.

I used the inbuilt DirectX Line class, which has a draw method that takes an array of points that make up a line, and draws rectangles between them.

I also drew a circle at each point on the line, so that the line was smoother (as the rectangles sometimes had gaps between them.

The following image shows the line tool at work. It was drawn using the Wiimote.

The width of the line tool can be varied by clicking the + and - buttons on the Wiimote.

I have also optimised the painting of shapes, and now only the most recent shape is redrawn on the canvas (this reduces the load on the processor considerably, especially when the image is large).

Line Tool

Hello again,

So today I have been working on the Line tool... This is different from the other tools so far, because each point on the line needs to be connected.

The Line tool is simply a list of circles, with the extra step of joining them together being done by the Lines Draw method.

I am only half way there so far. The line tool draws circles, and lines between the centers of the circles. In the complete version, a rectangle will be drawn between the two circles, so that the line doesn't have blots all over it (see image below).

I am still thinking about the best way to draw the rectangles between each circle. It is not that simple, as I must account for the angle between each of the circles in order to draw the rectangle correctly...

We will see what I come up with to do this.

Until next time...

Wednesday, September 17, 2008

Colour Selection Cube - Initial Attempt

So I have managed to create the colour selection cube in DirectX, and it looks pretty good I reckon. Here is a screenshot:

The colour selection cube is made up of 125 individual colour cubes, all drawn using DirectX graphics. Each cube consists of only 3 faces - the three visible ones - to make rendering faster.

A loop is set up in my code to create each cube, and give it red green and blue (RGB) colour values, ranging from 0 to 255.

To change to colour selection mode, the user clicks the 1 button of the Wiimote.

The goal of the next week is to get the Wiimote working with the colour selection cube, including using the distance of the Wiimote from the screen to determine the depth into the colour cube.

Also, this week I have to submit my usability study questions to the university ethics commitee, so that they can be approved by the time that the questionnaire needs to be done.

Will update my blog again soon.

Thursday, September 11, 2008

Colour Cube

Last week I handed in my partial treatise, with draft versions of chapter 1 and 2, and some content in the results and conclusions.

Now I am working on creating the colour selection cube in DirectX that will be used to change colours for painting.

Here is an example of what the colour selection cube could look like:

Friday, August 29, 2008

Partial Treatise

This week I have been working on my partial treatise, which is due on the 5th of September. The partial treatise consists of draft introduction and background chapters, and some content for the results and conclusions chapters (as much as is possible at this stage of the project).

I have managed to partially complete the first 2 chapters so far, and my aim is to have done this, and outlined my current initial prototype in the results section of the treatise by the 5th of september.

The introduction chapter (1) consists of a discussion of the project, the requirements and scope limitations, and a brief description of the contents of the thesis.

The background chapter (2) consists of a history of input devices (mouse, keyboard, graphics tablet etc...), an outline of current research into the Wiimote, and a discussion of the existing paint prototype, WiiPaint.

I have given my project a name, which is WiiPaintPlus. From now on in this blog, I will use this to refer to my work.

Saturday, August 23, 2008

WiiPaint Initial Prototype

So I have been making some good progress with an initial prototype, and have already produced much of the functionality that is present in the original program, in my new DirectX based program.

So far I have done the following:
  • Created a better mouse movement algorithm, so that the Wiimote can control the mouse cursor all the way around the screen (if the user is far enough away from the sensor bar).

  • Implemeted draw methods for a number of shapes in DirectX (Rectangle, Circle, and Spray Paint).

  • Made a button that clears the drawing panel.

  • Added a custom hand cursor (similar to the Wii menu cursor) that appears when the user is over the drawing panel.

  • Added several basic colours that can be selected to paint with.
So by controlling the mouse with the Wii, and selecing a shape and colour, the user can draw on the screen (by pressing the A button on the Wiimote).

Here is a video of this program in action:

Since the partial treatise is due in 2 weeks, I will be putting most of my effort into getting that done from now.

Sunday, August 17, 2008

Detailed Update


Over the last week, I have made some good progress on the project.
I have done more testing of the existing prototype created by a previous student, and have come to realise that I was using the system incorrectly in previous tests. The existing prototype does make use of the Wii sensor bar to pick up movement, and the mouse cursor can then be controlled by the Wiimote. This eliminated a number of issues that I had documented in the project plan:

  • There is a visible cursor for the user to see their current location.

  • The user has the ability to move the cursor to a new location and continue painting.
I would probably have realised this eariler, had I been more thorough in my reading of the existing project report, but having discovered this now, I have not lost much time at all.

Here is a video of me using the WiiPaint program:

As you can see, the pointing capabilities of the remote are still quite poor, and it is difficult to paint anything that resembles an object. Also, you can see that the colour selection tool is quite jumpy when switching between colour gradients.

DirectX Cursor Program

I have also been working on a simple DirectX program which has a cursor that is moved by the Wiimote. This is based on some of the projects by Johnny Lee (see link in the right menu), particularly the head-tracking project.

Here is a video of me using this basic program:

I will update more as soon as I can.