Avatars, Virtual Reality Technology, and the U.S. Military: Emerging Policy Issues
Avatars, Virtual Reality Technology, and the
U.S. Military: Emerging Policy Issues
Clay Wilson
Specialist in Technology and National Security
Foreign Affairs, Defense, and Trade Division
Summary
This report describes virtual reality technology, which uses three-dimensional user-
generated content, and its use by the U.S. military and intelligence community for
training and other purposes. Both the military and private sector use this new
technology, but terrorist groups may also be using it to train more realistically for future
attacks, while still avoiding detection on the Internet. The issues for Congress to
consider may include the cost-benefit implications of this technology, whether sufficient
resources are available for the communications infrastructure needed to support
expanded use of virtual reality technology, and whether there might be national security
considerations if the United States falls behind other nations in developing or adopting
this new technology. This report will be updated as events warrant.
Virtual Reality and Web 2.0
Virtual Reality (VR) technology enables Web content that is interactive and user-
generated. However, VR technology goes beyond traditional Web pages, and allows users
to operate in three-dimensions of space, simulated within the computer. VR is part of
what is known as Web 2.0, a second-generation method of using Web technology to
create communities, or social networks, where instead of passively viewing content, each
user can dynamically create and modify and share Web content. Thus Web 2.0
technology allows users to collaborate and create self-organizing communities that can
(1) increase the value and power of peer relationships and (2) simultaneously disrupt
traditional real-world methods for hierarchical control over information flows.1
Early examples of 2-D Web 2.0 social network communities include YouTube,
LinkedIn, and Wikipedia. Similar Web 2.0 tools were developed and used by junior
military officers during Operation Iraqi Freedom to create notes about observed enemy
movements and then quickly share this information among multiple unit commanders.
1 Don Tapscott and Anthony Williams, Wikinomics, Penguin Group, New York, 2006.
Initially, senior military officials became concerned about security and their loss of
control over classified information flows.2 However, unit commanders convinced DOD
leadership of the value of peer-controlled sharing information about insurgents’ adaptive
movements, which resulted in an officially sanctioned system now called Cavnet, which
allows fast access to knowledge acquired on the ground in combat.3 Other social
networking sites now sanctioned by the military include “Companycommand” and
“Platoonleader,” both accessible via password.4
Many businesses now use different 2-D Web 2.0 tools to accelerate innovation, and
several large corporations have also established simulated presences inside commercial
3-D VR worlds to expand real-world sales. Several government agencies, including the
Centers for Disease Control and Prevention, the National Oceanic and Atmospheric
Administration, and at least one member of Congress have experimented with setting up
their own presences in VR environments.5
Simulation and Military Training
Modeling and computer simulation have traditionally been used to train military
pilots and tank crews. In many cases, the trainee steps into a simulator device, which is
surrounded by screens that generate a 3-D image completely controlled by high-powered,
computerized artificial intelligence. However, new VR tools go beyond many traditional
limitations and now allow multiple participants to freely interact with each other in the
similar 3-D computerized environment. In many cases, computer networking capabilities
have dramatically increased to the point where users can now run complex 3-D VR
simulations in the field using a laptop connected to the Internet.6 Training with VR
allows large numbers of personnel to interact in a simulated face-to-face environment
with other distant military units through the Internet (or through the classified network
known as SIPRNET), and with first-responder units, civilians, and even medical
personnel units, providing a training experience that is increasingly effective, but at a
much lower cost than would be required for assembling these personnel for a real-life
face-to-face training exercise.
VR technology offers a potentially effective and economically efficient tool for
training military personnel to better deal with dynamic or potentially dangerous situations.
VR participants use self-designed computer images called avatars that look and act like
real people, and operate in virtual world environments that can have almost any
2 PBS Frontline, Innovating and Improvising, [http://www.pbs.org/wgbh/pages/frontline/shows/
company/lessons/].
3 PBS Frontline, Innovating and Improvising, [http://www.pbs.org/wgbh/pages/frontline/shows/
company/lessons/]. William Matthews, Helping Info Flow Freely Insurgents Outdo U.S.
Military, Says U.S. 3-Star, Defense News, January 28, 2008, [http://integrator.hanscom.af.mil/
4 Dan Baum, Annals of War: Battle Lessons, The New Yorker, January 17, 2005.
5 Trudy Walsh, Get a life: Congress, NOAA test the water in a virtual world, GCN.com, February
6 Kathleen Hickey, Virtual training gets real, Government Computer News, November 19, 2007,
p. 16.
combination of simulated characteristics — a busy checkpoint in downtown Baghdad, or
a subway tunnel in downtown New York under chemical attack by terrorists. The
resulting interactions can be unpredictable and can seem highly realistic to the individual
operators involved. Some of the avatars can simulate local citizens and, when operated
by real-life actors, can demonstrate culturally correct gestures, show facial expressions,
and communicate emotions, which military personnel trainees must interpret correctly
during these virtual world encounters.7 For a more detailed discussion of the effects of
virtual reality on trainees and operators, see “Your Guide to Virtual Worlds,” at
[http://www.pbs.org/mediashift/2007/10/digging_deeperyour_guide_to_vi.html], a
website that is affiliated with the PBS media enterprise.
Intelligence Operations
The intelligence community has started exploring for how VR communities might
become actual battlefields in the future, complete with cyber weapons for launching
attacks against terrorists or other enemies. Intelligence officials point out vulnerabilities,
because vendors for many commercial VR communities do not keep records of
conversations between avatars, or monitor how VR areas are used, some of which can be
protected by strong passwords. However, other officials also think this level of concern
is unwarranted, and that fears of threats to national security in VR are overblown. They
also indicate that, even though some users may engage in clandestine activity, some VR
products employ electronic tools that automatically look for risky behavior by avatars.8
The intelligence community has also indicated that it may initiate a study of the
behavioral characteristics found in virtual worlds, to identify emerging social, behavioral,
and cultural norms. The lessons learned would be used to help automatically detect
suspicious behavior and actions of individuals operating in virtual worlds that might pose
a threat to national security.9
Communications Infrastructure
Some experts expect that virtual reality and virtual worlds will have the same impact
on business and government entities as has the Internet itself. However, U.S.
communications infrastructure and acquisitions methods are relatively old, and many
Asian countries are well-networked with advanced equipment that may offer advantages10
that could allow them to establish the global standards for VR technology. China, for
example, has begun construction of a virtual world called HiPiHi, reportedly with
7 Michael Peck, Soldiers Learn Hazards of War in Virtual Reality, National Defense, February
8 Robert O’Harrow, Spies’ Battleground Turns Virtual, Washington Post, February 6, 2008, p.
D1.
9 Office of the Director of National Intelligence, description of Project Reynard, Data Mining
Report, February 15, 2008, p. 5.
10 Press release, Gartner Says 80 Percent of Active Internet Users Will Have A “Second Life” in
the Virtual World by the End of 2011, April 24, 2007, [http://www.gartner.com/it/page.jsp?id=
infrastructure that will support up to 75 million simultaneous users.11 Some observers
note that any country that succeeds in dominating the VR market may also set the
technical standards for the rest of the world, and may also own and operate the VR servers
that give them unique access to information about future global financial transactions,
transportation, shipping, and business communications that may rely on virtual worlds.
Some U.S. military and state government communications systems rely on
commercial products and leased communications lines for training for homeland defense
operations.12 However, a research survey in 2007 found that “American companies are
not poised to be the leaders in embracing Web 2.0 technology in coming years. If
anything, they’re looking like laggards. Leading the way are Indian firms, 80% of which
plan to increase their investments in Web 2.0 over the next three years, compared with
69% of Asia-Pacific firms, 65% of European firms, 64% of Chinese firms, 64% of North
American firms, and 62% of Latin American firms.”13 These observers question whether
the United States could adequately protect the security of its assets if future VR
communications standards and servers were controlled by another country, or possibly an
adversary country.14
DOD Programs
The military is reportedly planning to build a virtual world called “Sentient
Worldwide Simulation,” which will “mirror” real life and automatically follow real-world
events in real time. Some virtual world scenarios are also used to train medical personnel
for mass casualty events. To do this, VR hospital rooms are designed to duplicate actual
civilian or military facilities, and are populated with individually controlled avatars for
nurses and physicians, which interact with avatars of National Guard members and first
responders who must also locate and transport avatars of patients with different injuries.
Other uses include therapy treatment for military veterans suffering from Post-Traumatic
Stress Disorder (PTSD). The treatment objective is to help veterans deal with troubling
experiences in Iraq or Afghanistan by immersing them back into the sights and sounds of
simulated past battles. To provide a treatment, the therapist can manipulate effects the
patient sees in a simulation to best suit the needs of the patient. The repeated treatments
are intended to gradually reduce the experiences that trigger the trauma, and the memory
11 Personal conversation with U.S. State Department officials, 2008.
12 Personal conversations with National Guard officials, 2007. For example, “Guardnet” runs
over a Wide Area Network (WAN) that is leased from a commercial carrier. It supports data,
voice, and video for National Guard homeland defense and remote combat readiness training, as
part of the Distributive Training Technology Project. Geoff Fine, National Guard Units Train
Without Leaving Home, National Defense, February 2003, [http://www.nationaldefense
ma ga zi ne.org/issues/2003/ Nov/ National_Guard.htm] .
13 Two views of Web 2.0 in business, [http://www.roughtype.com/archives/2007/03/american_
compan.php]. How businesses are using Web 2.0: A McKinsey Global Survey,
[ ht t p: / / www.mc ki nseyquar t er l y.com/ a r t i c l e _abst r act _vi s i t or .aspx?ar =1913&l 2=13&l 3=11&
srid=9&gp=1].
14 Ramsey Shaffer, Baxter Technology Group, personal communication, January 30, 2008.
of the event may become tolerable. Early results from trials reportedly suggest that virtual
reality therapy is well-suited to a generation raised on video games.15
“Urban Resolve” was a distributed training simulation carried out in 2006 by U.S.
Joint Forces Command (JFCOM), Joint Experimentation Directorate, simultaneously at
the U.S. Army Topographic Engineering Center at Fort Belvoir, Virginia, and at the Space
and Naval Warfare Systems Command facilities in San Diego, California. The training
exercise used virtual world technology to help develop skills in urban warfighting
operations for future joint force commanders. The scenario, which involved more than
2 million individually simulated objects interacting in Baghdad in the year 2015, pitted
a U.S.-led coalition force against an adversary operating in an urban environment.
JFCOM is also planning to run a series of experiments in another virtual reality
training exercise, called “Noble Resolve,” which involves homeland security scenarios
that protect against a possible nuclear attack by terrorists. In one simulation, a nuclear
bomb will explode, and the virtual world will enable evaluation of collaboration among
local, state, national, and international officials. Government officials are seen as
directing first-responder and law-enforcement operations alongside military personnel in
a realistic VR environment. According to the trade press, JFCOM officials have also
indicated that they plan to build intricate VR simulations of many cities, with the final
goal of creating a fully functioning virtual model of the entire world, which will be known
as the Sentient Worldwide Simulation. This reportedly will become a “synthetic mirror”
of the real life that will automatically follow real-world events in real time.16
The Training and Doctrine Command (TRADOC) Project Office for Gaming (TPO
Gaming) is a new Army organization intended to integrate video game graphics into
training simulations for soldiers and small-unit leaders. The new office plans to develop
an Army simulation tool kit, scheduled for deployment between 2010 and 2015, that
allows end users to build and customized their own training scenarios without needing a
contractor to do it for them.17
Operational and Cost Areas
VR is not always the best choice for military training. For example, VR simulators
are usually located in comfortable environments, with air conditioning to cool computer
equipment, so the operators never get tired from running with a large backpack, or wet
and cold, or otherwise physically stressed as they would in a real-world training exercise.
However, VR offers the following advantages: (1) the training mission can be repeated
numerous times; (2) worst-case and emergency scenarios can be simulated without the
risk of injury to trainees; and (3) real military equipment is not used in VR, so costs for
equipment maintenance and fuel are eliminated. VR is increasingly seen as a useful way
15 Virtual Iraq — VR Based Therapy for Post-Traumatic Stress Disorder, Defense Update, 2005,
Issue 3, [http://www.defense-update.com/products/v/VR-PTSD.htm].
16 Josh Rogin, JFCOM brings virtual world closer to home, FCW.com, April 4, 2007,
[ h t t p : / / www.f c w.com/ onl i n e/ news/ 98147-1.ht ml ?t yp e=pf ] .
17 Michael Peck, Constructive Progress, Defense News, December 3, 2007, p. 32.
to supplement or reduce the number of real-world exercises needed to adequately train
military personnel.
Policy Issues
Some experts expect that global commerce will come to rely heavily on VR. In a
world where banking, transportation control, communications, and other global commerce
are arguably conducted in VR, they question how the United States will defend its
national assets if global VR standards are dominated by China, or if a majority of VR
servers are physically located in and operated by China.
Right now, the U.S. military does training in VR. Some ask whether a day will come
when the U.S. military might also conduct actual war fighting operations in VR to defend
U.S. assets and national security. Should research be funded to develop exploits specific
to VR that can be used by DOD?
How does the U.S. communications infrastructure to support VR compare with that
of China, or with other countries?
To what extent could VR help the intelligence community track possible terrorist
activity?
How do the costs and benefits of VR compare to the cost and benefits for traditional
military training exercises? Have there been evaluations of VR effectiveness? What are
the implications of the expansion of this technology to other military activities?