|VRML Special Interest Group from Software Forum:Creating Photo-Realism In A 3D Cyberspace
|by Sue Wilcox
As usual the SIG kicked off with announcements of general interest; free
T-shirts from Seybold, the co-sponsor of World Movers conference, proved so
interesting the audience was jumping up and down to grab them out of the
air. SGI, the other co-sponsor, offered a raffle prize of a free pass to the
conference to all those who'd come with a spare business card to toss in the
Bruce Damer, the organizer of the Earth2Avatars conference, due in SF
this October, invited those interested in participating to 'come on down'
and talk to him. Collaborators were also being sought by CoolWare - VRML
developers located in Palo Alto who are working on Virtual Palo Alto.
Following the merger of RealSpace and LivePicture the VRML creation dept. at
RealSpace is looking for people. Contact Peter@livepicture.com to learn
more. WorldsAway announced its looking for testers to join its open beta as
they prepare to move onto the Internet.
This month's presentations revolved around the issues involved in the
creation of photo-realistic 3D worlds on the Internet. This session was very
different from the last one which waxed philosophical. This time each
presenter came with news of cutting edge, pre-beta technology in development
as well as demos of neat tools.
Reid Hoffman, of the WorldsAway group within Fujitsu, declared himself a
'technoculturalist' and proceeded to look at cultural issues in photo-realism.
Fujitsu Software Corp. has a particular interest in the uses of
photo-realism. It has a tool for converting 2D data into 3D: RealModeler
(formerly known as SketchVision), and its own multiuser environment:
WorldsAway. Reid says, " the benefits of photo-realism are so obvious,":
the environment looks real, you can create virtual places and recreate
travel destinations and your avatar looks just like you so you are easily
But these benefits might also be a problem. What if the
definition of the world is too high? There are problems of addiction if
virtual worlds become too sensory and gripping. There could be problems with
motion sickness if the immersive effect is too great. Reid made a plea for
the virtue of avatar anonymity. The talking to a stranger effect produced by
wearing a mask is lost if your avatar looks exactly like you. And how do you
tell sites and people apart if the Taj Mahal can be exactly copied as a
virtual environment and your face can be worn by any avatar? Despite these
fears he said that WorldsAway intends producing some photo-realistic worlds.
He worries: " If the world becomes too exciting you might lose the person
behind the gloss" but its a problem I'd love to face.
Reid then moved on to demonstrate RealModeler. Working with a Japanese
interface --an American release has not even been scheduled yet-- he showed us
how easy it is to mark the major wireframe structure onto a 2D photo then
convert it into a 3D solid with the surfaces from the photo texture-mapped
onto the faces. There is even an auto-modeler using primitive shapes and a
library of shapes to speed up the model building, and an auto-texture mapper
to put the right image on the appropriate face of the polygonal structure.
"The idea is to have a very quick production process" Reid explained.
RealModeler is widely available in Japan -- if you can handle the Japanese
Sven Technologies President Saul Kato came to dazzle everyone with how far a
new company can come in less than a year since incorporation. He has three
products in development: SurfaceSuite, professional adaptive texture mapping
software for dealing with non-planar objects, AvatarMaker, a tool to enable
complete beginners at 3D to create their own avatars, and Digitact, a low
price simple to use 3D object scanner. He told us we needed to know how to
achieve "Kinesthetic -Immersivity" in our virtual worlds, then explained
"that means its gotta feel real" so you can lose yourself in the interface.
To achieve this you'll need to emulate real world motion, make sure there's
none of that irritating LOD-caused jumping about of objects and no popping
up of new objects when you reach their trigger inline boundary. Then you'll
need a smooth frame rate, not one that changes depending on the complexity
of the world, and of course textures on every face of everything. This left
everyone feeling 'yes, of course, but how?'. Its all very well to hold up
Quake as an example to us, its on a CD. The answer of course is to use
Sven's tools for applying textures.
Then Saul asked what's the best way to texture something. Not surprisingly
he believes photo-based texturing is the key to realism. He does acknowledge
its a high hurdle but says its one we need to jump as human faces are the
'epicenter' of recognition and expression and creating them takes up so much
time for professional 3D graphics artists. Mapping a photo to a facial
geometry is Sven's specialty. They are working on mapping to animated faces
All this is leading to custom avatar creation to get the mass market
involved in avatars. Saul believes that people need unique identity and that
their appearance is paramount to feeling an individual. So customizable
avatars are essential. But because its equally important to have an identity
that persists an avatar must work across time, platforms and multiuser
worlds. And identity should be more than reading a text label, facial
recognition is the key to knowing who someone is. So back we come to Sven's
technology being crucially important to making a good avatar.
Saul demonstrated AvatarMaker, the OpenGL version, on a laptop PC. Its
designed to require minimal user investment in money, labor or expertise
There are six 'stations' accessed from a central area where
you view your avatar as it is assembled. The 'BodyBuilder' is where you
choose from a display library of body parts, each of which is customizable
by using simple sliders to control complex parameters. (There is per axis
control over parts of the model.)
'FaceMapper' lets you choose a face from a
selection that at the moment ranges from Sandra Bullock to Chewbacca then
uses per vertex controls to autogenerate a set of textures based on control
points on the face and the underlying geometry. (The famous faces are
unlikely to persist in the downloadable demo and final release, due to
licensing issues. So the faces will be lesser-known male and female model
faces, as well as some fantasy animal/alien type faces. One day there will
be "avatar signature collections" to plug in to your FaceMapper.)
'Wardrobe' is where you choose ornamentation: from clothes to skin color,
while 'PropShop' has a choice of accessories ranging from hats to guns.
'Pose' crashed. Its "a little buggy right now" Saul confessed but managed to
get it on its feet again pretty rapidly. It controls stance and attitude of
the body, and as it has joints and articulation it makes setting a figure in
a particular pose very simple. (Future plans include using it as an
'Teleport' is the last of the six 'stations'. Here is where
the export of your completed avatar is handled.
The beautiful 9,000 odd
polygons of the model will have to be decimated to make it acceptable in a
VRML world although Sven have plans to distribute AvatarMaker on a CD-ROM
and hence provide client-side storage of bodies. If you have this library
available on users machines then file sizes are only 2K and information
passing over the Net only has to pass in 8 real-time floats to manipulate
texture coordinates. Teleport has a "great down-res feature, so it can
on-the-fly reduce LOD to match the requirements of the multiuser service
providers" says Saul.
He's tested AvatarMaker on all the major systems and
its output is compatible with them all. [Clarifying later comment from Saul:
"We have indeed tested AvatarMaker to produce output compatible with the
major virtual world systems' avatar formats --Worlds, BlackSun, Oz, Sony,
The Palace [bitmap snapshots], any VRML2.0 based. But the current versions
of some of these (Worlds, Oz, Sony) don't allow custom avatar import so
we'll have to see if and when they add this functionality.
will be output-ready so if the companies maintain a semblance of their old
formats it will work, and we'll certainly push our hardest to make future
versions/downloadable patches ensure direct compatibility. We have worked
quite a bit to get to know the internals of each system so this will work."]
Saul intends to bundle the client-side software of all multiuser companies
that accept Sven avatars, as part of the CD-ROM. A 'ZAP' feature launches
any selected virtual world application from Teleport so users never have to
leave the illusion of a room. Sven is hoping to link up with games companies
to produce or adapt a multi-user Doom style product with fully customizable
AvatarMaker should be available in March for around $50. A beta program
should be starting very soon, check Sven's Web site, and strategic partners
are being sought to help roll the product out. Sven's future plans include
building behaviors into AvatarMaker, adding powerful new modeling,
sculpting, breeding ? and 3D scanning tools and enabling natural person to
person interaction by providing synched facial expression, speech, gesturing
and body movement using motion tracking combined with mapping moving textures.
Peter Hughes, Live Picture's VRML Texture Mapping man, answered the question
"Why Photo-realism?" with a simple answer: "because we want virtual worlds to
be huge and an important part of people's life." He had us suggest what is
the competition for time spent in virtual worlds. It seems the biggest
competitor, apart from real-life in general, is TV. Therefore to beat TV and
suck people in he believes we have to make photo-realistic worlds.
describes worlds out on the Internet as "still plastic for the most part" by
which he means they lack textures. The textures that are used are very
limited:128x128 pixels, 8 bit colors, in a palette that doesn't match the
screen palette, and with seams if you try to tile a texture larger than the
allowable size. He calls this 'Texture hell' and says the only browser not
to suffer from it is RealSpace's Real VRTraveler. He examined the question
of how to get photo-realism in an online virtual world. At the moment if you
take a 2,000x2,000 pixel texture from a photo and tell your computer to map
it onto a 3D form you can "leave it for up to a week" to render. Not the
path to real-time performance! So the answer is to take the textures and, in
a sense, remove the resolution from them. Then you have resolution
independent textures or multi-resolution textures.
LivePicture use their
FlashPix open file format to achieve this effect. In essence they make each
layer of the image half the resolution of the previous layer. Tiles are used
to achieve this: a 1,024x1,024 pixels image has 128 tiles, 512 square has 64
tiles, 256 square has 16 tiles, 128 square has 8 tiles and the limit is a 68
square with only 4 tiles. Each tile is a JPEG image. Using this method you
can ask for a specific area of a picture and only get the relevant tiles. To
request tiles you have to use the Internet Imaging Protocol also developed
by LivePicture. These technologies are all going to be integrated with Real
VRTraveler to make photo-realistic VRML worlds possible.
Peter's demo was of a panoramic scene set in Scott's Antarctic hut as
abandoned in 1906. He showed us a conventional zoom within the room and the
resultant pixelation. Then he showed us a zoom into a FlashPix image on the
wall and let us compare the resolution and clarity of the image. FlashPix
Queen Victoria was clear and sharp while JPEG Albert was a fog of undefined
pixels. This 'partially populated panorama' also contained a 3D box that was
constructed using FlashPix tiles so any part of it could be examined in
detail. Its quite something to be able to wander into Scott's hut in the
snow and be able to pick things up off the shelves and read the labels.
LivePicture are currently working on combining their technology with that of
Sven to produce high-resolution human heads.
There was the usual cut and thrust of questions after the presentations. But
they covered such a range of topics, from VRML 3.0 to copyright issues on
star's faces that I'll have to leave you to watch the video of the
proceedings. Next time just be there. Its going to be: Addressing the Need
for Speed with SGI and Newfire (formerly Axial Systems) and will be held at
SGI's offices at 2021, N. Shoreline Blvd. Mountain View CA.
The VRML SIG is sponsored by Fujitsu Software Corporation through the
Software Forum To find out more contact Katherine Bretz at
firstname.lastname@example.org or see the Software Forum.
Videos of the proceedings are available for $25 from Fem Energy Box 1176
Boulder Creek CA 95006, (408) 338-7228 or email@example.com
Usually found online at inquiry.com as VRMLPro,
regularly writes on VRML and 3D graphics for Web
publications. In her spare moments she is a game
interface designer and co-author of a series of books on oriental game
theory and Go.