Faculty Survey

Presentations, Publications, and Reports

The Academic Technology Landscape: Examining Faculty Use and Views at Arizona State University

The Applied Learning Technologies Institute alt^I

Laura Brewer, Ph.D.
Director of Research & Evaluation
Applied Learning Technologies Institute
Arizona State University

Zeynep Kilic, Ph.D.
Senior Researcher
Applied Learning Technologies Institute
Arizona State University

Introduction

Innovative technologies are integral to the effectiveness of faculty of the New American University. Technology can facilitate, enable, and transform faculty instructional and research activities. Likewise the effective use of technology directly impacts the experience of our students. We, as an institution, establish and promote a culture of communication and collaboration, where technology is the platform for connectedness and outreach. The Applied Learning Technologies Institute (alt^I), of ASU's University Technology Office, is dedicated to the advancement of higher education through research, collaboration, and action. The Institute brings together faculty, researchers, students, and professionals, working hand in hand with programmers, engineers, designers, and support staff, toward a common goal; ensuring the success of all learners (see UTO strategic plan).

Faculty face several instructional technology challenges today. Adopting technology is a process that involves time - initially to discover the technology, to investigate and learn the application, later to effectively use, and finally to see results. Rapid changes in technology, balancing the time needed to learn to use the tool or application with the time needed to accomplish the task itself, may discourage or dissuade faculty. Often times newer technologies are introduced separately and not integrated into existing Learning Management Systems already familiar to faculty. To invest in the newer tools requires additional time and leads to technology management concerns. Just as technology is rapidly changing, so are student preferences and expectations. Institutional decisions to switch products or discontinue support for certain tools may create an unstable and unpredictable environment for faculty. Faculty may decide that investing time when the future is unpredictable may be too costly (McGee & Diaz, 2007). Understanding the issues surrounding technology integration and innovation from the faculty's perspective can help institutions find solutions that will yield results and benefit the entire campus. Gathering data from faculty to inform decisions around academic technologies is critical.

Toward gaining a better understanding of access to, use of, and interest in technology for their work, The Applied Learning Technologies Institute conducted a faculty technology survey at the end of the spring 2007 semester. In particular, we were interested in the ways faculty currently integrate technology into instruction and their interest in learning more about how to utilize academic technologies. Data from the Spring 2007 Faculty Survey is considered in connection with findings from a similar study of ASU student technology use conducted by alt^I in Fall 2006. These surveys are intended to serve as a part of an ongoing, multi-faceted assessment protocol that can influence and inform technology adoption, design, and support at ASU, throughout the community, and across disciplines.

Methods

The ASU Spring 2007 Faculty Technology survey consisted of 211 closed-ended and 11 open-ended items organized by focus into six sections: 1) Experience with Technology, 2) Teaching Internet, Hybrid & Web-enhanced Courses; 3) Innovative Curriculum; 4) Disaster Planning - Continuing Instruction; 5) Assessing Student Learning with Technology, and 6) Background Information.

Data were collected using an online survey administered between April 17, 2007 and April 27, 2007 to 4,370 faculty members. For this study, "faculty" were defined as anyone serving in a "teaching" role during the spring 2007 semester - this includes full-time and part-time faculty, instructors, lecturers, graduate assistants/associates and administrators or staff who were teaching courses. The initial invitation to participate was sent by email on Tuesday, April 17, 2007 and reminders were sent to those who had not yet responded on Tuesday, April 24 and Thursday April 26, 2007. Overall, 1,846 faculty completed the survey--a 42% response rate.

A web-based survey application, SurveyMonkey, was used to conduct the data collection. There were no systematic technical issues with the data collection. Individuals with questions or concerns about the study were prompted to email an internal address. Email sent by survey participants received a prompt, personalized response by alt^I research staff. This strategy of personalized interactions with potential non-responders is known to improve response (Cook, Heath, & Thompson, 2000; Dillman, Tortora, & Bowker, 1998). Data was cleaned and coded for quantitative analysis that was conducted in SPSS. In addition, qualitative analysis was conducted of the open-ended responses. Tables reporting univariate percentage distributions for all closed-ended survey items can be found in Appendix.

Respondents

Introduction

Who responded to the ASU Faculty Technology Survey? In effort to examine how well the faculty survey respondents represent all ASU faculty, the demographic distributions were considered and respondents' self-report was compared to institutional data from the 2006-2007 ASU Fact Book.

The overall distribution of Spring 2007 Faculty Technology Survey respondents was similar to the existing distribution of faculty across ASU campuses and colleges (see Chart 1). Some colleges are somewhat over-represented in the Spring 2007 Faculty Technology Survey (i.e. Education) and others are under-represented (i.e. Engineering). But overall the distribution of faculty response across the colleges is similar to the distribution of all faculty by college.

Chart 1.
All ASU Faculty and Faculty Technology Survey Response Distributions by College

Again, the Spring 2007 Faculty Technology Survey aimed to gather response from all faculty, staff and graduate students who were in a teaching role. Approximately two-thirds of all respondents reported being in 100% full-time equivalent (FTE) positions. Of the part-time participants, three-quarters were graduate assistants/associates and faculty associates. In contrast, 65% of the full-time respondents were ranked faculty members. The distribution of ranked faculty response is similar to the distribution of all faculty by rank (see Chart 2). One difference, however, is that the survey respondents appear to under-represent full professors and over-represent "other". This pattern may be a result of using self-report scales for rank on the survey. A number of full professors identified as "other" on the survey because they were concurrently in administrative positions (i.e. Department Chair, Director, etc...).

Chart 2.
All ASU Faculty and Faculty Technology Survey Response Distributions by Faculty Rank

Although there are proportionally more men in faculty and graduate teaching positions at ASU, women were more likely to respond to the Spring 2007 Faculty Technology Survey (Chart 3). Some researchers have found that women are more likely to respond to online surveys (Sax et al., 2003; Underwood et al., 2000 cited in Lippert, 2003). Yet, this pattern is not consistent across all surveys. Other research has found that men are more likely to respond to web-based surveys (Carini et al., 2003; Tomsic et al., 2000 cited in Lippert, 2003; Kehoe & Pitkow, 1996; Palmquist & Stueve, 1996; Smith & Leigh, 1997). It is important when interpreting faculty response to keep in mind that women are slightly over-represented in this study.

Chart 3.
All ASU Faculty and Faculty Technology Survey Response Distrubitions by Gender

Demographic information about the respondents is compared to the population to determine to what degree the two groups are similar. The more similar the respondents are to the population with regards to demographic items, the more confident we can be that the respondents represent the population on all measurement items. Examining representation across colleges, faculty rank and gender suggests the respondents are similar to the overall population of faculty. However, we should consider the fact that some colleges, men, and full professors may be under-represented in the data.

Access to Technology

Many perceive limited access to computers and the Internet as a common obstacle to technology use, especially in education (the Digital Divide Network, n.d.; Hawkins & Rudy 2007). However, almost all ASU faculty report they have access to computers at home (97.6%) and at work (99.4%) [1]. In addition, most ASU faculty have a high-speed Internet connection from home (85.8% overall - 26.9% DSL and 58.9% Cable). Similar findings about higher education faculty having access to computers and the Internet have been reported elsewhere. For example, the National Center for Education Statistics reported 10 years ago, in fall 1998, that 97% of instructional faculty and staff at degree granting institutions had Internet connectivity at work and they were also likely to have access at home. Although access may be considered an issue for other populations, within higher education in general, and among ASU faculty in particular, access to computers and the Internet does not appear to be the barrier.

What types of computers do ASU faculty use? At home and at work, more ASU faculty use PCs than Apple Macintosh computers (Chart 4). However the proportion of Apple users at ASU is significantly more than the company's reported market share of 4.8% (Dalrymple, 2006). To support a consistent and effective learning environment across vendors, ASU's 1:1 Computing program promotes a select group of suppliers, including Apple and Dell, to offer faculty, staff and students affordable options to buy or lease computers. This approach is in line with ASU's goal of providing a customized teaching and learning experience for each individual user.

Chart 4.
Type of Faculty Computer User by Location

ASU faculty are also more likely to use desktop computers than laptop computers, especially at work (Chart 4). In addition, many faculty who own or use laptops also own or use a desktop computer. In contrast to faculty, many ASU students report [2] they only use laptops and fewer students report owning both a desktop and laptop computer (Chart 5). Using laptop computers represents a shift to mobile technologies. ASU is continuing to increase the level of wireless access across its campuses, which supports mobile technology use.

Chart 5.
Faculty and Student Computer Ownership/User

Mobile technologies are increasingly the delivery platform of choice for students (The New Media Consortium, 2006), and faculty are lagging behind in this area. Some faculty go as far as prohibiting laptop use or wireless access in their classrooms, which exhibits a gap between what faculty and students perceive as useful and necessary technologies for education (Efaw, 2005; Alexander, 2004).

"Students views of what is and what is not technology are increasingly different from those of faculty. From small, flexible software tools to ubiquitous portable devices and instant access, students today experience technology very differently than faculty do, and the gap between students view of technology and that of faculty is growing rapidly." (The New Media Consortium, 2007: 4).

Even though their students are more likely to prefer mobile computing, nearly all ASU faculty (91%) use a desktop at least some of the time. Using a computer fixed to a particular location does not appear to have a negative impact on faculty satisfaction with their computer access. When ASU faculty were asked whether they are satisfied with the computer access they have for their work, 96% of all respondents indicated they are at least somewhat satisfied and 78% said they are satisfied or very satisfied with their computer access (Chart 6).

Nationally, there has been a trend of increased satisfaction among faculty with regards to their computer access. According to findings from the 1998 National Study of Postsecondary Faculty (NSOPF), 46% of full time faculty rated their institutions' quality of computing resources as good and an additional 1/3 rated those resources as excellent (Warburton et al., 1999). In a more recent NSOPF study conducted in 2003, 78% of faculty reported being very satisfied or somewhat satisfied with the quality of equipment and facilities available for classroom instruction (NCES, 2004). Though faculty satisfaction with computing resources, including classroom technologies, has been increasing nationally, in our study ASU faculty report even greater rates of satisfaction (96% reporting at least somewhat satisfied see Chart 6).

Chart 6.
Faculty Satisfaction with Computer Access

Unlike faculty, who appear to be content with their computer access, students exhibit lower levels of satisfaction regarding campus technology (Hartman, Moskal & Dziuban, 2005; Oblinger 2005). Digitally literate students from the Net-Generation come to college with high expectations about the availability of innovative technologies; however, their colleges struggle with cycle of adopting and integrating the newest technologies (Calhoun, 2004). This challenge is not unique to institutions. Faculty are also lagging behind in the integration of technology into their curriculum. Keeping up with this kind of pressure leaves faculty feeling "like a Pony Express rider when the telegraph came along" (Gustafson, 2004: 40). A January 2005 ABOR report on Arizona's educational future identifies this discrepancy between students' expectations for academic technology use and the choices their instructors make. "The in-coming students grew up in this environment. The educational methods in most universities do not take full advantage of the many combinations of technology and the changed student" (ABOR, 2005). The generational differences between faculty and students are striking. Faculty are not as highly immersed in the rich digital environments that attract and engage their students, which makes the academic technologist's job challenging (Campbell et al., 2007).

Applications and Activities

How do ASU faculty currently use technology? What applications do they use? Nearly all ASU faculty report using email (99%), Web-browsing (93%) and word processing (93%) at least once a day or more often. In contrast, many faculty report they have never used Eportfolios (66%), Google Docs (60%) or video conferencing (55%) (Chart 7). When looking at technology and web-based, Web 2.0 activities (i.e. blogs, wikis, digital audio, digital video, podcasts), more faculty report being users of content than producers of content. For instance, nearly all ASU faculty indicate they have never authored a blog (80%), never contributed to a wiki (81%), nor produced a podcast (89%) and most faculty report they have never referenced a wiki (59%), never produced digital audio (64%), never produced digital video (64%), nor listened to a podcast (55%) (Chart 8). Web 2.0 technologies are considered at the forefront of popular technology trends, and they are especially relevant for education (New Media Consortium, 2007); the ASU findings are important in that they illustrate how faculty technology use is inconsistent with contemporary developments, as well as with student expectations and experiences.

Chart 7.
Frequency of Faculty Use of Applications

Chart 8.
Frequency of Faculty Activity

Although ASU faculty are behind their students with regards to technology use, they are ahead of their national counterparts in some ways. For example, the National Study of Postsecondary Faculty reported 69% of full-time faculty using email in 1998 (Walburton et al, 1999). In fall 2003, 72% of instructional faculty across the nation reported they used email to contact students (NCES, 2004). In our study, 92% of ASU faculty are using email for instructional purposes (Chart 9). Given that the large scale national data available only addresses faculty use of an older technology - email, one can imagine how far behind the national faculty might be in terms of newer, much more interactive and collaborative Web 2.0 tools such as wikis, blogs, and podcasts.

In addition to email, many ASU faculty are using other technologies for instruction. For example, most faculty report using word processing (89%), web-browsing (85%), and presentation software (81%) to teach and many faculty indicate they use spreadsheets (63%) for instruction (Chart 9). Certain technologies have penetrated our daily lives so widely that it is not surprising to see these technologies used for many purposes (personal, research, administrative and instruction). In addition, other technologies are designed for particular purposes; in these cases, we expect faculty to report applying the technologies to specific purposes. For example, most faculty indicate they use presentation software primarily for instruction (81%) and research (55%), less for personal reasons (23%). Whereas faculty are more likely to use spreadsheets for personal use (41%), perhaps as a result of the relevancy and intended functions of the software.

Other technologies are less widely used by ASU faculty. For example, 40% of faculty report they have developed web-pages for instruction. It is likely that the wide-spread use of learning management tools (i.e. Blackboard) by colleges and universities facilitate the production of these instructional web-pages. Yet, less than 25% of faculty indicate they develop web-pages for other purposes (personal, administrative, or research). In general, the fact that fewer faculty have experience developing web-pages for any reason represents the trend that most people are consumers of technology rather than producers of content on the Web.

Chart 9.
Faculty Use of Applications by Purpose

As discussed above, most ASU faculty, though highly computer literate, are not well versed in newer, interactive and collaborative applications (e.g., blogs, wikis, digital audio/video, or podcasts). Yet there is a small group of ASU faculty who are using these Web 2.0 technologies...but for what reasons? Among those faculty who report using these technologies, more indicate they do so for personal reasons (Chart 10). For example, 36% of faculty who read blogs do so for personal reasons compared to 23% who read blogs for research, 17% for instruction, and 7% for administrative purposes. And 23% of faculty who listen to podcasts do so for personal reasons, versus 14% for instruction, 13% for research and 5% for administrative purposes. Perhaps there is a progression from using technology for personal reasons to integrating technology into teaching and research. As these technologies become more familiar, perhaps using the applications in an educational context is more likely. However, such a progression would require faculty see an educational value before they begin incorporating these technologies into their curriculum. We will discuss in later sections how institutional policies and faculty training might affect these trends.

Chart 10.
Faculty Activities by Purpose

Chart 10 shows a slight preference among ASU faculty for blogs and podcasts for personal consumption, while using digital video and audio are more (or as) likely to be used for instructional purposes. ASU faculty are also reading (again consuming) blogs and wikis for research purposes and perhaps diffusion into the instructional realm is the next step. This transfer into instructional use, at least for blogs and wikis, requires faculty to become content producers with a new technology and enable their students to do the same. However, faculty could incorporate other technologies, such as digital audio/video and podcasts, into their curriculum without being content producers. Either case requires faculty to have a basic understanding of the "how to" as well as a pedagogical understanding about "why" to incorporate such technologies into their teaching. Although interactive and collaborative Web 2.0 technologies are being used more frequently, and may be more attractive to students, many faculty have not attempted to use and most have not yet tried to integrate these technologies into their instruction.

Summary and Implications

Teaching and Learning with Technology

Introduction

Shifting to a student-centered learning environment is not easy for all faculty. For example, one faculty respondent reported: "I've had students mention that Blackboard was old technology and that better options are available although I don't know of any myself." The digital divide between students and faculty seems to be getting larger with older faculty, in particular, wanting to stick to a brick-and-mortar style of teaching values. Introducing technology for its own sake is problematic; yet faculty are increasingly reporting that students want more online activities incorporated into their classes (Allen & Seaman, 2003). Moving from a traditional teaching style to one that incorporates Web 2.0 enhanced pedagogy is like looking at the learning journey in two distinct ways: do students "take a guided tour or go backpacking on their own?" (Laandpere cited in Miller, 2007). However, moving from classroom lectures to teaching in an online environment requires a transition and preparation phase. Faculty might feel incompetent, may not have enough time or support and may be unsure how this will improve their teaching (or student outcomes) (Littlejohn et al, 1999; Hartley et al., 2005). These days it is widely accepted that pedagogy is more important than technology (the tool) itself (Brewer, 2001; Byun et al., 2000; Liu, 2001; Palloff & Pratt, 2001), and faculty naturally want to be convinced that these technological additions will indeed improve student learning and outcomes. It is helpful to consider content, pedagogy and technology as interconnected rather than being independent constructs (Koehler et al, 2007).

Administrative support, institutional policies, organizational goals, university culture and the availability of technologies influence the choices faculty make around using instructional technologies (Bullen & Janes, 2007, Wyman & Stringfield, 2006). Moving from a teacher-centered to a learner-centered environment requires pedagogy focus more on participatory and collaborative knowledge building. These approaches involve connection, community and collaboration be equally important as the content (Conrad, 2007). A curriculum that takes advantage of interactive technologies through gaming, simulations, problem based learning, collaborative knowledge creation (such as wikis) will contribute to student learning and better address different learning styles (Connolly & Stansfield 2007a). While technology can improve the learning process, it is critical that its use be informed, otherwise it simply becomes a hindrance (Vallance & Towndrow, 2007). So, one challenges is to encourage faculty to see that technology can and will enhance their teaching and their students' learning and satisfaction if a sound pedagogy is applied.

In this section, we begin with a presentation of current teaching with technology practices at ASU and the pedagogical implications of these approaches. We follow with a discussion of training and support for instructional technologies. And we conclude with a consideration of the role of assessment with technologies.

Current Practices and Pedalogical Implications

"[...] CMS are not typically used in ways consistent with deeper learning principles, but rather are primarily used as a means of information dissemination. ... An often overlooked but significantly important influence on the teaching and learning process is an instructor's belief or conception about teaching and learning. These conceptions have a significant influence on the instructional practices or strategies they employ" (Apedoe, 2005: 58-59).

There are many ways faculty can utilize technology to improve and support their teaching. Such efforts may range from developing and utilizing digital copies of notes and presentations to integrating innovative technologies, like podcasts, blogs, wikis or e-portfolios, into courseware. Today's students are looking for digital content that is rich, dynamic and interactive, which is quite different than the traditional printed textbook (Nelson, 2006). With the increased adoption and use of web-based learning management systems (i.e. Blackboard) that require content to be presented digitally, we would expect more faculty members to become skilled at developing their own digital content or locating and using existing digital resources. At ASU, faculty use of digital content is divided with about one-third of faculty reporting most (75% or more) of their course content already in a digital format, another third indicating some (25% to 74%) of their content in a digital format, and the final third report having little to none of their content in an electronic or digital format (Chart 11).

Chart 11.
Percent of Curriculum/Course Content Currently in an Electronic or Digital Format

Many faculty are not digitizing their curriculum due to lack of time and resources, as well as to concerns about available content not being applicable to their preferred pedagogical approaches (Harley et al., 2006; Hartley et al., 2005; Littlejohn et al, 1999). A curriculum that incorporates technology, however, is no longer simply a trend, but a requirement (Ouzts & Palombo, 2004). A large number of ASU faculty are already on the right track by having digitized a portion of their curriculum; yet, nearly one third are still only teaching with traditional content. This is important to consider when developing initiatives that require content to be in an electronic or digital format, such as online courses or programs. In addition, institutional resources and policies must assist faculty in the process of moving their content to a digital format or locating quality digital resources. An increased focus on digitizing curriculum is necessary as more courses and programs are offered online.

According to the Sloan Consortium's 2006 study of higher education institutions in the southern United States, over 99% of the larger enrollment schools have some type of online course or program offering (Allen & Seaman, 2007). Online offerings are disproportionately available at the larger, public, doctoral/research institutions, like ASU. Yet, the National Study of Postsecondary Faculty, which surveys faculty every five years, reported little change in the percentage of faculty teaching Internet or distance learning courses - only 6% in 1998 and 7% in 2003 and 40% of faculty teaching web-enhanced courses in 2003 (Bradburn, 1999; NSOPF, n.d.). In stark contrast to the national statistics, at ASU 70% of faculty reported having at least some experience with developing or teaching Internet, hybrid or web-enhanced courses [3] (Chart 12). The high percentage of experienced ASU faculty may be a result of college or departmental initiatives requiring courses have a web presence. Compared to their national counterparts, ASU faculty seem to be ahead with regards to transferring their curriculum into a digital format and by trying a variety of online teaching approaches.

Chart 12.
Faculty Experience with Developing and/or Teaching Internet, Hybrid, and/or Web-enhanced Courses.

Arizona State University, like other colleges and universities across the United States, has utilized a commercial learning management system (LMS) for about a decade. Today, faculty at ASU can choose among a number of learning management tools and approaches ranging from a commercial product - Blackboard; to the open-source collaboration and learning environment for education - Sakai; to myASU - a personalized, web-based environment that integrates various tools including, ASU Interactive services, gmail, Google Documents/Spreadsheets, newsfeeds, myASU courses, etc... EDUCAUSE reports that more than 90% of all campuses surveyed had a LMS in place (Hawkins & Rudy, 2007). Yet, the majority of institutions use a single commercial LMS (66%), 6% support an open source LMS and less than 10% are like ASU and have a hybrid approach (commercial and homegrown or commercial and opensource) (Hawkins & Rudy, 2007) . Thus, ASU is atypical among higher education institutions with its integrated strategy.

The EDUCAUSE 2006 Core Data Survey (Hawkins & Rudy, 2007) found over 70% of faculty used a LMS selectively, meaning they were not required by their institution to use a particular tool for teaching. Almost all ASU faculty (94%), who reported they had at least some experience with developing and/or teaching Internet, hybrid and/or web-enhanced courses, indicated they use Blackboard, even though they have other options (Chart 13). This pattern is likely a result of ASU having a relatively long institutional commitment and history with Blackboard, as well as many faculty not being aware of other available applications for teaching online. In addition, although nearly one quarter of faculty reported that they use their "own website" at least once a week for online teaching and learning, it is likely that many of these instructors are referring to their Blackboard class websites. We know that developing and maintaining a personal website requires a skill set that most faculty do not possess. While only 8% of ASU faculty indicate they have used the Sakai Online Collaboration and Learning Environment (CLE), the open-source alternative to Blackboard, we would expect these numbers to increase as more faculty become aware of this application.

"For the next-generation systems to be truly effective learning environments, faculty must move from easy-to-use online organizers to effective pedagogical tools for assisting and assessing desired learning outcomes" (Carmean & Brown, 2005: 5)

Chart 13.
Frequency of Use of Tools to Teach Internet, Hybrid, and/or Web-Enhances Courses.

Blackboard is currently the LMS of choice by ASU faculty. Of those faculty who indicate they had experience teaching Internet, hybrid, or web-enhanced courses, a very large proportion, 81%, report they currently use Blackboard. However, the ways in which faculty use Blackboard varies widely (Chart 14). For example, there are a number of features that most faculty never use, such as the virtual classroom (71%) and safe assignment (67%). There are other features that have not been explored by a substantial proportion of faculty, such as groups (37%), assessment tools (35%), and digital dropbox (28%). Finally, there are a few features used by almost all faculty, such as the gradebook (85%), adding announcements (99%) and adding content (100%), which are the most basic functions of the LMS. These features involve faculty input only, meaning students can view the information in these areas but they can not contribute. This one-way (faculty to student) communication and presentation of material reflects traditional classroom instruction. In contrast, the more interactive features that require a shift in pedagogy and student participation, like the virtual classroom, groups, and assessment tools are less used. This discrepancy in what features instructors choose to use exemplifies that most faculty using Blackboard have not adopted a more interactive pedagogy that can be made possible with the technology. Instead most faculty are sticking to what they know, which is presenting information to the class as the content expert and therefore reproducing their traditional instruction in an online environment (Apedoe, 2005; Kuriloff, 2001) . Vodanovich & Piotrowski (2003-2004), for example, reported that though most of their respondents believed Internet to be an effective instructional tool, less than half actually utilized it in their instruction. This discrepancy between behavior and attitude probably exists across higher education institutions.

"For the most part, the tools that exist [in current CMS] are designed to support the generic activities of teaching with technology (i.e., quizzing). Among technologies missing are tools that support the varied subjects and teaching styles that comprise the full constellation of instruction that exists"(Pittinsky, 2003 cited in Apedoe, 2005: 64)

Chart 14.
Blackboard Features - Percent Faculty Never Use

One exception is the high proportion of faculty who report using the discussion forums (81%), which can involve interaction and engage students in innovative learning experiences. But, faculty may also use discussion forums as another way of presenting their content and not necessarily to include student participation. Research shows that learning differs online when compared to similar techniques face to face (Curtis & Lawson, 2001). In addition, it is important to keep in mind that using a particular feature does not guarantee a successful teaching and learning experience. Since this survey did not focus on teaching practices, we do not know how faculty used each specific feature. However, literature on effective online learning environments stresses the significance of the pedagogical choices instructors make in connection with their curriculum and technology choices (Brewer, 2001; Pelz, 2004). For example, Bill Pelz (2004) suggests three principles of an effective online pedagogy: 1) Let the students do the majority of the work, 2) Interactivity is central, and 3) Aim for presence (social, cognitive, or teaching).

Just as faculty decisions to use particular Blackboard features does not result in successful teaching and learning, use of a particular technology does not necessarily result in faculty satisfaction. However, we did find a relationship between the extent to which ASU faculty use a Blackboard feature and their level of satisfaction. For example, those features that are used by nearly all faculty also have the highest levels of satisfaction, i.e. adding announcements (80% satisfied) and content (79% satisfied) (Chart 15). For most of the other features, there is a moderate level of satisfaction. This pattern is likely a result of many faculty not knowing how to effectively integrate the tool into their instruction, which can lead to frustration and disappointing teaching experiences. Again, teaching online successfully requires a shift in pedagogy, which many faculty have not yet attempted. Instead, many faculty try to use the technology to replicate their traditional classroom teaching. Faculty satisfaction is notably lower with the virtual classroom (44% satisfied) and digital dropbox (39% satisfied) perhaps as a result of technical shortcomings with these features.

Chart 15.
Satisfaction with Blackboard Features Used

Faculty who reported they currently used Blackboard were also asked to provide additional comments about their experiences with the tool. In short, faculty remarks fit into three thematic areas: 1) the technology's functionality, 2) examples about how and why they use Blackboard, and 3) their need for additional support and training. Please see examples in Table 1.

Table 1.
Current Blackboard Users - Additional Comments

1) Functionality

"There are several things I'd like to see improved, including being able to support cross-listed courses, being able to set a release time for discussion forums (like you can for everything else) give feedback in the dropbox and gradebook, download batches of documents from the dropbox, have automatic shut-off times for quizzes, etc."

"When you teach multiple sections as I do (4 or 5 every semester) Blackboard is a pain...to use. The interface is extremely clunky requiring too many clicks to do practically anything and everything. A front end local client with drag and drop capability would be a start. My non-response to an interest in learning more should be interpreted as little interest in learning what BlackBoard can do now; that I know. What I would be interested in learning about are NEW capabilities that would make me more productive as well as my students."

"Blackboard is simply a poorly-designed system. It takes too many steps to get anything done, and wastes more time than it saves. I build a course website on my personal space for each class I teach because it's far easier and faster than using Blackboard. I use Excel to build gradebooks because Blackboard's gradebook is not only unreliable, it limits how I can grade assignments. The only thing I will use Blackboard for are the Discussion Boards, and even those are a pain to use, taking too many (repetitive)steps to get anything done. It's simply a poorly-designed system."

"BB is adequate for posting materials, but seems overly complex for doing some simple tasks. I think more work needs to be put into customizing it to have a simpler work flow for common tasks. For instance, posting reading material involves several steps when this should, by default, be a simple one-click process. If one wanted to do something more complex, those options should of course remain available, but I am sure many go unused by most. I also do not feel very connected to BB. I find its interface frustrating and slow, and this dissuades me from very frequently checking discussion lists or groups I have been associated with. I believe having having a front page that one could see at a quick glance all new activity would be helpful. I also think this should be better integrated with myASU for this purpose as well."

2) Examples of Current Use

"Blackboard is very useful for teaching. It has essentially displaced the use of transparencies."

"Overall, I feel I've been able to compensate for some of the shortcomings through face-to-face time. Since I will be teaching online soon, I think my use will change quite a bit."

"I think the reliance upon these technologies is disappointing because it is diminishing the face-to-face interaction that is the heart of an educational experience. Additionally the structure of many of these tools compromises content in lieu of efficiency. I only use the blackboard for large lecture classes where it is essential to manage communication with students and grading."

"I really like using Blackboard. I often teach large sections with between 100 and 400 students; Blackboard is a great way to maintain consistent contact with students. I also like that I can post PowerPoint slides to download during class lectures from Blackboard and that I can make these available only to me and my T.A.s."

"Blackboard is occasionally awkward and frustrating, but nevertheless it is one handy tool for a hardworking teacher! It would be difficult now to imagine the classroom of 400 students without it!"

3) Training and Support

"It is just hard to keep up with all that Blackboard can do and the changes that continue to happen. I use Blackboard every semester, but I always feel behind the curve. Plus, I hate how limited I feel using Blackboard, but don't feel I have the time (and perhaps talent and resources) to learn how to make it be more responsive to my students' needs."

"would like to use more effectively/consistently; open to training on this!"

"I've taken a week long training class given by faculty development or what ever it is called, (the class was MARVELOUS), but what I need now are some "booster shots", a way for a quickie update of skills, or better yet, a person I can call, or drop in on, for a quick lesson, however I'm on Polytechnic which makes it more difficult"

"Much of what I've learned has been through my own experiments--I'd like to learn more about the applications from an expert"

"Faculty should have administrative staff who can support these functions. It is to much for me to do when I am responsible for research, teaching and service activities"

"I've found support of blackboard wanting here at asu. when I've contacted support directly about a gradebook issue, i was told to google my question, as many OTHER schools have very well-developed blackboard sites. seems that if we're going to pursue this digital experience, we need much improved support for faculty."

"When learning it takes a lot of time to set up the features. Up till now I did not have enough lead time to set up the templates early enough to be ahead of the course timetable. Once it is set up, it is fun to use. I would love to be able to have someone else set up the program, so I could just monitor and implement its use."

In addition, faculty who indicated they were not currently teaching with Blackboard were asked about why they had chosen not to use the tool. Their reasons for not using Blackboard fell into four distinct categories: 1) they are not teaching at this time, 2) their course content doesn't lend itself to Blackboard, 2) they are using another technology, and 3) their need for training and support. Examples of faculty comments can be found in Table 2.

Table 2.
Currently Not Using Blackboard - Additional Comments

1) Not Teaching

"Not teaching a course this semester for ASU"

"I am not teaching this semester, otherwise I would be using it."

2) Content

"The course I taught this semester was a laboratory course. It doesn't lend itself as well to Blackboard."

"I teach art classes and the hands-on nature of the course does not lend itself to the blackboard environment."

"Lack of time I do not believe it is neccessary for my course."

"My instructional style depends on personal interaction.

3) Using Something Else

"Currently using MyMathLab with courses taught - which has everything blackboard has to offer plus connections to textbook - with help for students."

"I use WebWork, the Math & Statistics Dept. online homework system instead. Blackboard was too hard to use (5 years ago when I stopped using it) for incorporating math symbols into teaching. And problems only came in one version. Soon I will probably switch to another online course management system."

"We are giving a stand tool called taskStream which is used by all accredited institution for offering Education Technology to pre-service teachers. This lets the government measure the perforamance of the students using TaskStream."

"I have used TaskStream for the past 6 years. I beleieve it is a better fit with my teaching, assessment, and program evaluation than Blackboard."

"I work with Herberger Distance learning, who tailor-make course to suit my instructional goals and needs."

"No need. Math Department uses WeBWork for homework and I update my website with class information as needed."

"I use Sakai"

4) Need Training and Support

"Juse never taken the time to get geared up for it. Began one time, but dropped the action."

"I don't feel competent enough."

"I use MyASU courses to pull up my roster. Otherwise I don't really know what Blackboard is for. No one has ever explained it to me."

"Lack the training and time to teach myself."

"I have attended an introductory course, but need more one-on-one instruction to understand the process"

"It is not the "skill", it's the practice that is crucial to good teaching. ASU does a poor job of creating instructional community, sharing of good ideas and effective course design, etc. What happened to the teaching excellence committee?"

"I've found support of blackboard wanting here at ASU. When I've contacted support directly about a gradebook issue, i was told to google my question, as many OTHER schools have very well-developed blackboard sites. seems that if we're going to pursue this digital experience, we need much improved support for faculty."

Looking across the two groups (current Blackboard users and those not using Blackboard at this time) we find a couple of interesting patterns. First, some current users are able to articulate pedagogical implications and examples of where Blackboard can improve the management of large courses. Also, current users are more likely to discuss and describe shortcomings in Blackboard. In contrast, non-users do not focus on Blackboard's weaknesses; rather they are more likely to discuss other tools that better meet their content requirements (i.e. MathLab for math symbols). Some of those faculty who are not using Blackboard appear to be complying with individual departmental or program initiatives regarding academic technology use, instead of making a personal choice. Finally, both groups of faculty indicate they need additional training and support, which we discuss in detail below.

Regardless of level of technology use, preferences for various tools, or experience teaching online, all ASU faculty indicate a moderately high level of interest in learning more about how to teach various types of online courses (Chart 16). However, more faculty report being at least somewhat interested in learning about web-enhanced teaching techniques (81%) than other types of online learning that result in a loss of face to face classroom instruction, like hybrid teaching (74%) and Internet courses (63%). As discussed earlier, initially faculty feel most comfortable integrating technology into instruction when they are not required to change their pedagogical approach. Many faculty have questions about if, and/or how, technology can improve teaching and learning. Their suspicion, hesitance, or lack of enthusiasm might stem merely from lack of knowledge about how to effectively teach online. Thus, one challenge institutions like ASU face is to provide training and support for those who are interested in learning more, while also trying to reach out to those who are not aware of the contributions technology can make to improve their teaching. Additionally, the institution must work to sustain faculty interest in adopting newer technologies and innovative pedagogical techniques over time.

Chart 16.
Interest in Learning More about Developing and Teaching Internet, Hybrid, and/or Web-enhanced Courses

Training and Support

Regardless of the approach, institutions must make the necessary human and financial investments. Faculty development for existing and future faculty is a pivotal investment for integrating technology in higher education; it can catalyze innovations in learning across generations. (Moore et al, 2005)

While most colleges and universities offer some type of support for faculty using academic technologies, there is not agreement about how to best provide these services in a manner that satisfies institutional goals as well as faculty, staff and student expectations. Closing the gap between student expectations and faculty expertise requires institutions address three major areas: 1) awareness of student needs and preferences regarding technology, 2) enabling faculty through training and support, and 3) the integration of pedagogy, design, institutional policy and support (Moore et al, 2005). The EDUCAUSE 2006 Core Data Survey (Hawkins & Rudy, 2007) found that support level and availability varied across type of higher education institution. In particular, doctoral degree granting institutions provide more support than MA granting institutions, which provide more support than BA and AA degree granting institutions.

Almost all institutions struggle to design faculty development that can rapidly adapt to technology and personnel changes. Today's faculty workforce is more diverse, with more adjunct and part-time instructors, which has an impact on an institution's ability to provide support. Not everyone in a teaching role can come to a physical workshop (Mangan & Patel, 2005). Types of faculty support and training can vary not only by delivery mode, but also by who offers the services. Two of the most widely used support methods are training upon request or scheduled workshops (Hawkins & Rudy, 2007). At ASU, academic technology support and training is offered online, over the phone, and face to face, and is available through central institutional resources as well as individual academic units.

Of those faculty who reported having some experience with online, hybrid or web-enhanced courses, 70% indicate they had used some type of Blackboard/myASU course support. Over half of the experienced faculty reported using central, campus-wide resources such as myASU/Blackboard online and the myASU helpdesk line at least once a semester (Chart 17). In contrast, less than half of the faculty report they had used support from their local department (40%) or college (34%) at least once a semester. A moderate proportion of faculty indicated they had never used any available support (central or local), which may mean that the technology worked smoothly for these faculty and they never needed to contact support staff. However, another possible explanation may be that these faculty did need some level of support/training, but were not aware that services were available.

Chart 17.
Faculty Use of Various Blackboard/myASU Support

Overall, most faculty who have used the various types of Blackboard/myASU support report they have been at least somewhat satisfied with the services (Chart 18). Most faculty report being very satisfied or satisfied with the Online Helpdesk (77%) myASU/Blackboard online resources (76%), myASU telephone support (73%), their college level support (71%), their department technology support (66%), and their campus technology support (63%). Nationally, when asked about the institutional support for implementing technology-based instructional activities, 84% of higher education faculty reported to be at least somewhat satisfied (NSOPF, 2004). A higher proportion of ASU faculty report being at least somewhat satisfied with support in comparison to the faculty nationally.

Chart 18.
Faculty Satisfaction with Various Blackboard/myASU Support

In order to understand more about faculty experiences with support, we asked faculty to provide additional feedback about their use of Blackboard/myASU support and services. We found that findings from faculty open-ended responses regarding their use of myASU/Blackboard support revealed four major themes. Faculty are: 1) satisfied with support, 2) confused about or unaware of available support, 3) prefer a particular type of support, or 4) they have difficulties with support (Table 3). Those faculty who report difficulties indicate problems in three areas: 1) communication, 2) timing, and 3) staff expertise (also Table 3).

Table 3.
Faculty Feedback Training and Support

1) Satisfied with Support

"The Helpdesk people helped me fix my problems."

"I go to our campus support person. She is very knowledgeable, understands what an instructor needs (pedagogical issues), and takes it on herself to cut through the bureaucracy. For example, if I mess up my course shell request, she will fix it rather than just refer me to someone else. That's the kind of service that we all appreciate."

"The workshops were very useful. The instructor OUTSTANDING!!!"

2) Confused about/Unaware of Available Support

"I am unaware of a department technology support group."

"Confused about how college/campus/department support works together..."

"It's confusing about how to find help. I had found one person who I could call anytime to ask questions but he left his job. I haven't found someone to replace him as my Blackboard answer person."

"Other than my department's technology support group (English), I was unaware that any of these other support groups existed. Again, as I mentioned earlier, perhaps these forums could be made more visible and links/phone numbers/locations more accessible."

"I am not aware of the difference between "campus technology" and "college technology." The support I rely on regularly is provided by the tech support unit in COOR [building]"

"I didn't know that these support groups existed. Are you sure they do? Maybe I didn't realize that they existed. Don't know."

3) Prefer a Particular Type of Support

"I very much like having personal departmental faculty support that comes to my office. I also like being able to go to see a person when in the Computer Commons when I'm having major problems with Blackboard."

"I use the people I already know and have established relationships with in the distance learning group (ASUOnline)."

"I usually call a friend to get help"

4) Difficulties with Support - Communication Problems

"It's awful. Why can't we train technical people to speak and write in meaningful ways to non-technical people they are supposed to help? Why do I have to understand their IT terminology and accept their arrogant attitudes? Since I am the one being supported, should they try to understand me and my goals a little bit, rather than me having to operate on their turf? And they just love to make us fill out forms. They are so operationally driven that it disgusts me. It's all about them and their processes and forms."

"Helpdesk folks are generally responsive and helpful but on weekends things can be slow and when the error message says "Contact System Administrator" and you click on that button it can take hours to get a response... I think that button should automatically click through to send a trouble ticket and not just send an email....Online faculty resources are obtuse...at least the ones in the manual...I tried using it a bunch at the beginning of the semester and gave up. You waste less time with trial and error."

5) Difficulties with Support - Timing

"When dealing with one person they should follow through with that person until the issue is resolved and not take other calls. Also, the hours are terrible. They close at 4pm and the university goes on into the night with night courses so something needs to be worked out. Too often all the people that work there that have the answers or can answer questions (especially MAC questions) are always at lunch. Please hire more talent and more people. This is a large enough campus to justify it."

6) Difficulties with Support - Staff Expertise

"Course support staff is always prompt in returning responses to questions. Via the phone they are also very polite and accommodating. When the problems are complex, then the satisfaction level goes down fast. With my limited expertise I need to figure out how to solve problems with e-mail. In a very inelegant manner I have had to have students send their messages to a third party who in turn forwards them to me! The problem has been ongoing over at least 3 semesters and several classes."

"My experience with centralized help desk is extremely not satisfying. It takes sometimes a week and a lot of exchange of e-mails to resolve a problem that can be resolved in 3 minutes. I have a feeling that the organization of the help system considers more convenience of its own staff, and not the convenience of the end user. Several times I had the feeling that students assisting at the help desk are simply not competent. It happened couple of times that I was told that the problem cannot be resolved or the assistants didn't know how to resolve it, while a freshmen student of computing science from China was able to resolve the problem in one minute."

"Some technical assistance is pretty good and some is not. It depends on who you get. I have found they can't really handle complex problems. I had to hire an outside expert to fix my Blackboard problems this semester."

"My responses are somewhat misleading. I frequently left dissatisfied because the staff can't help with problems that are often beyond their control - outages during key periods of time, problems with the BlackBoard design, etc."

Note: Central HelpDesk services were reorganized Summer 2007.

A number of ASU faculty reported that they had difficulty with technical support because of they could not communicate effectively with the support staff. Bower (2001) argued that fear of appearing incompetent is a significant barrier for faculty who are accustomed to being the experts. Other ASU faculty indicated they were looking for the support process to be expedited and that support be delivered consistently. Faculty perceptions of the available technology support, and the quality of this support, are critical to their decisions to integrate technology into their instruction.

Lack of adequate training is a major barrier to getting instructors involved in online learning (Galusha, 1998; Bower, 2001). Lee & Busch (2005) found that instructors' willingness to participate in distance education depended on their perception of the adequacy of training they received as well as recognition they get as a result of their participation. Support and training are especially important when teaching online, since in this setting faculty are not only the content-experts but also technology specialists for their students (Lee & Busch, 2005; Fernandez, 2001; Sherry 1995; Xu & Morris, 2007).

In 1999, UCLA's Higher Education Research Institute (HERI) found that faculty ranked keeping up with information technology more stressful than research, teaching and publishing demands. Some have found that training is best delivered hands-on to acquaint instructors with current technologies (Sherry & Morse, 1995). Also, because it takes more time to develop materials online, a sustained interest and commitment from the faculty is paramount (Alford & Engelland, 2001). If tenure and promotion criteria include, and reward, such faculty endeavors, we are more likely to see faculty pursuing innovative online teaching methods or devoting more time to technology integration (Bower, 2001; Murphy & Terry, 1993). [Similar findings are also discussed in Kang, 2001; Luck, 2001; Torrisi & Davis, 2000; White, 2000].

Chart 19.
Faculty Interest with Various Support Methods

According to Efaw (2005) a successful faculty development model should include "peer-to-peer training on available technology; modeling of successful classroom techniques by more experienced instructors within the discipline; support, accessibility, and feedback from a trusted mentor; and established forums and mechanisms, both within departments as well as school-wide, that enable all faculty members to share ideas and continue development." However, without sufficient budget set aside for training and support of faculty, the money spent on technology implementation is wasted as the majority of the faculty would be less inclined to incorporate it into their teaching (Briggs, 2006). In such cases, only a few, with intrinsic motivation and enthusiasm, may drive the technology incorporation as isolated attempts. We must take into account that a large portion of higher education faculty did not choose their profession because of a strong interest in technology. However, they now find themselves in a rapidly changing technology environment to which they are expected to adapt which may cause some of them to feel like the "Stranger in a Strange Land" (Hartman et al., 2007: 1). Hence, we need to take the diversity of technical know-how and interest among faculty into consideration. The support and training models should include a variety of techniques to bring in faculty with different preferences and availabilities. This idea of varied techniques was also one of the findings in our survey. Regarding support preferences, there were no clear winners. ASU faculty reported moderate levels of interest across different support methods, with phone and f2f modes garnering higher levels of interest. Only 48% of ASU faculty say they are interested or very interested in web-based seminars. Yet 61% are interested in online tutorials and demos. This may be due to a perceived difference in synchronous/asynchronous modes.

Lastly, it is important to recognize that technology integration requires a shift in faculty working style. Faculty usually act as lonely individuals in their teaching role where most, if not all , decisions, preparation and execution are accomplished by themselves without too much input from others. When working with technology, they have to incorporate others such as instructional technologists, help-desk and support staff, trainers and even students (Hartman et al., 2007). This is a new way of operating for them which requires an adjustment in their work styles.

Assessment with Technology

Accountability is a major concern in higher education and demonstrating student success has become a central focus. Thus, colleges and universities have placed a greater emphasis on systematic and transparent assessment approaches, involving improved data collection techniques and reporting strategies, with technology units becoming central to this process (Campbell et al., 2007).

Stewart et al. (2006: 201) mentions the main difficulty of assessment for online programs as cheating (student identification), which they solved by creating test items that targeted higher levels of knowledge and application. To this end, there are ways in which we can utilize technology to assess student performance. For example, if ongoing assessment of student performance is linked to immediate feedback and individualized instruction it has been shown to support learning (Riccomini, 2002; Kashy, et al., 2003). Swan (2004) recommends providing frequent opportunities for testing and feedback as well as automating these feedback loops when possible. However, instructors may be utilizing online components in terms of the content, but may be hesitant to use online assessment techniques to avoid issues like cheating and even consider limiting use of technology in classroom setting to prevent intellectual dishonesty (Hartman et. al, 2007).

We asked ASU faculty about the importance they assign to using technology as part of their assessment strategies and if they were interested in learning more about it. A large majority of our respondents are at least somewhat interested in learning more and they also believe in the importance of areas we have asked (Chart 20). Our questions included use of technology to show student mastery of course objectives where 82% felt was important or somewhat important while 80% were interested in learning more about it. Additionally we asked how faculty felt about actively including students in self assessment which 82% reported to be important while 79% indicated that they want to learn more about it. Also asked was the use of technology to link assessment activity to course goals, of which 80% found important while 78% reported interesat in learning more about it. Finally, in responding to a question about using technology to link assessment activity to larger program goals, 75% reported they find it important and 72% indicated interest in learning more about it.

Chart 20.
Perceived Importance and Interest in Learning More in Using Technology for Assessment Techniques

We also asked ASU faculty about their preferred assessment techniques in an open ended question. Respondents listed the expected, more traditional techniques such as essays, journals, reflection papers, exams, quizzes and oral presentations. There were also unique techniques listed, such as video, rubrics, direct observation, project/problem based learning, portfolios, student self assessments and capstone. Unfortunately there are also some ASU faculty who feel rather negative about technology and prefer face-to-face instructional methods. Interestingly, assessing with technology does not imply that teaching/learning can't take place in traditional ways. Many of the techniques can be facilitated and improved using technology. Again faculty would need training and support to understand how this can be achieved and the advantages associated with it. Table 4 includes a selection of quotations from the open ended question about preferred assessment techniques for measuring mastery of learning outcomes. Some ASU faculty outright reject that technology can be a useful instrument for student assessment. Other faculty specifically find the existing technology to be lacking (such as Blackboard shortcomings). Table 4 also include examples of how ASU faculty are using technology for assessment and finally what kind of technology they envision for assessment.

Table 4.
Preferred Assessment Techniques

1) Technology cannot aid in assessment of learning.

"I teach writing. I know of no substitute for a close reading of student essays and of offering students timely feedback on their efforts. I know of no machine that can perform this function"

"Not the sort of assessment that computers accomplish efficiently. Authentic assessments are tasks where students incorporate learned knowledge and skills in projects. Such projects can be displayed using technology."

"Currently, essay-style tests and activities. Multiple-choice is also an option, but there are too many security issues with those (do students take "team" tests? can they print the questions and then give them to classmates? and so on...)"

2) Blackboard is lacking in assessment

"Since most of my courses require critical thinking, the only real way to assess that is through essay exams-- and Blackboard does not do them well."

"One disadvantage BB's fill-in-the-blank function was impractical. Because of my humongous student load, reading every single quizzie was out of the question; I needed BB to generate a score without stumbling over misspellings and upper- or lower-case punctuation. So, I had to restrict BlackBoard quizzes to true-false-multiple-guess questions. "

3) How is technology utilized for assessment by ASU faculty

"My students have a digital portfolio they create. I then upload the portfolios to Blackboard for all to share."

"Online exams and quizzes that are somewhat different for each student, i.e. problems with randomly generated data that results in different correct answers."

"My preferred assessment is lab work turned in, however, students are able to perform this activity by utilizing web/internet resources without actually doing the work. As a result, quizzes are a necessary evil."

"I used BlackBoard's "quiz" function in the true-false-multiple-guess mode as a teaching tool and to persuade students to study reading assignments related to specific skills I needed them to master before they began their writing projects. Students took a number of BlackBoard quizzes during the first half of the semester; quizzes were designed to highlight the most important issues I wanted to bring to the students' attention and to allow students to apply learned facts and skills to new examples. A secondary value of the quizzes was that they allowed students to build a "fund" of points toward the semester grade that would buoy their final score even if their basic writing skills were somewhat wanting."

"WebWork-based homework, progress checks and pre-exam worksheets. While the interface is not as elegant as Blackboard, it is free and provides more than enough tailoring capabilities to fit course/student needs."

"I teach writing, so the student submits his/her paper to Safe Assignment, then I download it to make comments using "Track Changes" and drop it into his/her Digital Dropbox. I think it's hard to use technology-based assessments given my subject area."

4) How is technology envisioned for assessment purposes by ASU faculty

"I'd love to have access to the remote controls (not sure of the name) that students can use in the classroom to answer questions."

"A student-compiled eportfolio would be ideal. I'm working on ways to do that without Blackboard. I'd like students to have something of their own to continue making contributions to even after they leave my class -- and more broadly than that: after they leave ASU. This "show knowledge" material could aid them in applying for jobs, graduate school, professional school, etc."

"Now use tests, which is not the best. Interactive designs with feedback would be good. They do not exist for most of the material in the courses."

Additionally, faculty were asked to describe their expectations from ASU in supporting their efforts to integrate technology into their assessment activities. Responses fell into six general categories (Table 5): As in the question above, some ASU faculty insist that use and incorporation of technology does not guarantee better assessment. Overall, this group seems to feel defensive towards "more" technology and they are not convinced that technology by itself produces better learning outcomes. Rest of the faculty have raised similar needs throughout the survey; that they need more training, assistance and instruction (on assessment tools in particular), access to resources (such as a survey tool for faculty use), strategies and techniques to manage and facilitate grading, and finally, the need for improved security to decrease academic dishonesty.

Table 5.
How can ASU support assessment efforts to integrate technology?

1) Training

"Have more training workshops, webcourses, and tutorials available."

"A big help would be informing me of other models and what methods have been successful."

"Provide a variety of ways to utilize different assessment activities that would employ visual and verbal applications of concepts and criteria."

"More training in web-design, podcasts, and other new technologies."

2) Support

"Again, technical support that is available, responsive and integrated within the units is key, as is regularly updated and replace hardware and software."

"I just need to learn more about how it works--and that takes time I don't feel I have."

"Assuming that one has the skills to implement, the greatest support would be giving faculty the time to do it."

3) Security

"Academic integrity, technology ethics, and security support is critical. It is much easier for students to attend to "game" the system when using online learning. Mine - actually try to send me blank or bogus submissions - in an attempt to get "the computer" to give them the correct answer. They learn quickly that I still read their work! Implement methods to imprint student submissions so that the same material cannot be submitted by multiple students. At least they would have to retype it - then maybe they would learn something as they enter each word!"

"A better plagiarism detection systems that allows more interaction and viewability."

"Be able to demonstrate that the given student actually did the work - validation."

"Provide a place where my students can take examinations on a computer and also providing staff to manage the room to ensure that academic dishonesty is minimal."

4) Accessibility/Availability

"1. Support open source software that can replace or work in place of (or beside) expensive proprietary software. We need to help students be able to afford their education and also reduce software cost at ASU. 2. Provide information to students about open source software that can replace MS Office etc., that does the same thing but costs little to nothing. 3. How does the above help assessment? The students need the tools first. I teach a GIS course where we require an expensive GIS software package that ASU has a site license for. However, it is very tough to get ASU IT to provide student access to this software outside of the specific course instructional laboratory. Thus, students have nowhere to go to do class projects, since the computer lab is in heavy use and unavailable outside of their own class time. The lab is locked at all other times. 4. I teach many classes that require software. I have found that if I can use an open source software package that the student can also take home, they are likely to do better with the course material because they have access to the specific software they need whenever they want to use it. The high cost or limited availability of proprietary software is a limiting factor in our educational programs."

5) Discipline Specific Concerns

"ASU should recognize that many academic programs don't benefit from True Outcomes, etc. There is no advantage to our flight students to have work (portfolios) on file. "

"It is hard to integrate technology into the assessment of learning outcomes in art. If ASU can develop a computer that can recognize a good painting, then we might be talking."

"I have little hope that technology will have any serious impact on the interaction between student and teacher through the exchange of comments on student writing. I use track changes in Word to comment on student papers. That hasn't changed in years."

6) Technology does not mean better assessment

"All questions presume the benefits of integrating technology? This question presumes my agreement with that first presumption. I am not convinced that my teaching would be made Better for Students by the addition of Technology(other than in the case of a disaster that prevented any teaching from taking place in person--)"

"Because so much of the learning that happens in the classroom can't be quantified in such a way that it can be assessed using technology, it's extremely important not to reduce a university education to rote memorization and multiple choice exams. We need to continue to challenge our students thinking and learning. Furthermore we need to insure that our assessment tools allow for critical thinking and thoughtful responses."

Summary and Implications

Results show that 1/3 of ASU faculty already have most (75% or more) of their course content digitized, while 1/3 have some digital content (25-74%) and the remaining 1/3 report little to no digital content. Main reasons for not digitizing course content include lack of time and resources, and concerns over a mismatch between the content (the field), pedagogy and the available technology. The majority of ASU faculty (70%) have at least some experience with developing or teaching Internet, hybrid or web-enhanced courses. Almost all (94%) ASU faculty who teach such courses reported using Blackboard, though Sakai is also available at ASU. The ways in which faculty use Blackboard, however, vary widely. A number of pedagogically enhancing interactive features are yet to be explored by many faculty, such as the virtual classroom, groups and assessment tools.

In terms of satisfaction with the features, participants reported highest satisfaction with those features that are used by most faculty, such as adding content or announcements. Lowest satisfaction was reported with virtual classroom and digital dropbox, possibly due to technical shortcomings of these features. Faculty reported some technical, software and network issues that make it hard to work with Blackboard, however, there were many who also reported that teaching large classes with Blackboard makes course management much easier. Overall, faculty report a need for more Blackboard training and support. Those who are not currently teaching with Blackboard reported four reasons: currently they are not teaching, course content is not conducive to using Blackboard, they are using another technology instead, and they need more support/training before they can teach with Blackboard.

In terms of support available on campus, 70% of faculty teaching with technology reported that they use available technical support. Faculty use central, campus-wide resources more so than their local support staff (at the department or college level). Most support users are at least somewhat satisfied with the services they received. Again, more faculty reported satisfaction with the campus-wide support staff than their local technology support. Open ended responses show that a number of faculty are unaware of available support or prefer different kind of support than what is available. Those who have encountered difficulties with the support available listed communication, timing and staff expertise as three areas that need attention.

In the area of assessment, most ASU faculty (82%) think that use of technology for assessment purposes and actively including students in self-assessment are important . They also want to learn more about these techniques (79%). When asked about preferred assessment techniques, respondents indicated two distinct viewpoints: While some faculty think that technology cannot aid in assessment of learning, others think the existing technology (Blackboard) at ASU is lacking. When further asked about how ASU can support faculty in this area, responses fell into six categories. Mostly, training and support are needed for integrating technology. Additionally, to address cheating and student identification concerns, an accessible and secure assessment environment is needed along with assessment resources such as software. While discussing role of technology specifically in assessment, faculty, once again, expressed previously discussed philosophical viewpoint that technology does not mean better assessment. Finally, some faculty indicated their strong belief that their discipline does not lend itself well to such technology applications. These strong negative perceptions about technology might be a barrier to training efforts and need to be considered carefully.

In short, compared to the national average, ASU faculty are doing slightly better in incorporating technology into their teaching. However, adequate and available training and support are salient issues. Keeping up with a constantly changing landscape of instructional technology is a stressful and confusing job for many faculty, who are used to being experts in what they do. Trying new technologies might make them feel like novices, rather than experts, especially when their students seem to know more about the technology than they do, which results in a perceived shift in balance-of-power (Hartman et al., 2007). Helping some of the hesitant faculty see the real-life application in discipline specific ways, perhaps through peers rather than instructional technologists, and demonstrating value of technology integration in assessment is a challenge the institution is facing. Effective communication with the instructional technology support staff is paramount to gain faculty buy-in. However, it is also imperative that institutional tenure and promotion criteria recognize the time, effort and resources required to create and update a technologically enhanced curriculum.

Current Iniatives

Colloboration Models

Collaborative working models are becoming more a global norm with emerging technologies enabling and facilitating team-work processes. As the industries expect employees to flourish in team-based work environments, higher education institutions adjust by revising their curriculum. Group assignments, peer-review feedback, and collaborative learning activities are commonplace in many instructors pedagogical toolbox today. There is also another philosophy and movement thriving, and finding especially a welcomed place, in academia: the Open Source movement. The movement started in 1997 among software developers by making software code free and open to public and therefore created a process of systematically harnessing open development and decentralized peer review to lower costs and improve software quality" (Raymond 2001,

This idea of developing a product and a process collaboratively and making it freely available to the public is particularly an attractive one in an educational setting. After being funded by the William and Flora Hewlett and the Andrew W. Mellon foundations in 2001, for example, MIT launched in 2003 their OpenCourseWare initiative where all course materials were made available to the public (currently more than 1,700 courses (Brown & Adler, 2008)). Content was free but not necessarily collaborative is this true?-Zeynokilic 3/13/08 10:00 AM . ASU took the idea further by making the curriculum as well as the instruction (?) a collaborative effort. In Fall 2007, a new one-credit signature course called ASU 101 was introduced to all first-time freshmen to inform them about the unique opportunities at ASU. In the ASU 101 model, curriculum is developed in a collaborative team approach with faculty as content experts working alongside instructional designers to create innovative learning materials. Videos, podcasts, electronic documents, library resources, presentation slides, communication tools, and other material are designed and integrated into a digital resource library for faculty to utilize for instruction.

In this survey we asked faculty about their interest in using curriculum created collaboratively as well as their willingness to be a part of such collaborative initiatives. In ASU, there is moderate interest among faculty for using content produced by other ASU faculty or being part of a team that produces such content (Chart 21). However close to half (47%) of ASU faculty do not want to participate in a team curriculum effort at all, with 30% expressing no interest in using digital curriculum created by their peers.

Chart 21.
Faculty Interest in Using Content Built by a Curriculum Team and BUilding Digital Content as Part of a Curriculum Team

One explanation for the disinterest could be related to discipline specific concerns, as some ASU faculty reported in previous sections that their content does not lend itself to technology incorporation. An open-ended question prompted faculty to report any areas in their curriculum that might not readily translate into a digital format (Table 6). Answers included previously stated concerns about technology replacing face-to-face interaction and instruction, along with concerns specific to disciplinary fields. Additionally, some ASU faculty reported not being able to envision their content in a digital format and therefore reporting that such initiatives would not apply to them. Finally, ASU faculty indicated that copyright issues might be a barrier for digitizing content. Similar findings have been reported by Harley (2007) about why faculty do not use digital resources available to them. The foremost reason mentioned was the discordance between available material and faculty's teaching approaches, along with lack of time.

Table 6.
Areas Faculty report not being able to translate into digital format

1) Copyright issues

"Advanced literature seminars utilizing contemporary texts are often not available in digital format, and fall under copyright with regard to creating digitized versions of them."

"I use images almost exclusively as course content and am concerned about downloading them, their quality (this must be the best resolution) and copyright issues, even if used for course work."

2) Discipline specific concerns

"Certain courses are focused on face-to-face interaction and group cohesiveness (e.g., small group communication) or on presenting material before a live audience (e.g., public speaking)."

"Producing on air television newscasts - requires a TV studio and lots of students to man technical positions. Studio work such as practical performance studies have to be done in person. I also believe that teaching methods courses need to be embodied and face to face. In addition, many learners ( including adult learners) are not visual learners or audio learners, but learn best kinesthetically, which means they learn through DOING, not by just reading or hearing."

"Great deal of it. We are training music therapists. You can't learn music skills digitally and you can't learn how to interact with clients digitally. Pure didactic course may fit but there are few of them in my degree program."

"I teach electrical engineering. I do not think that there are any substitutes for hands-on hardware labs."

3) Technology should not replace f2f interaction

"Certain courses are focused on face-to-face interaction and group cohesiveness (e.g., small group communication) or on presenting material before a live audience (e.g., public speaking)."

"Yes, years of experience that are communicated through delivery of course content that expand, provide examples, respond to student questions and feedback in a dynamic, live exchange of ideas, synthesis, integration etc... all the higher level learning. there is a real danger that the means (technology) are starting to define the ends (course content)."

"I cannot digitize spontaneity and passion.

"The give and take between a professor and his students can never be fully translated into digital format. There's no substitute for talking directly to students."

We also asked about the types of digital content faculty is already using at ASU to identify a baseline of practice on campus. Faculty report that they use publicly available and free resources most frequently (on a weekly basis, 35%), followed by their own content (28%) and content produced by publisher (20%) (Chart 22). However 20% of ASU faculty indicated they never used freely available resources and 41% indicated they never produced their own digital content. Perhaps involvement in a collaborative model such as ASU 101 is one way of engaging faculty to consider digitizing their content since they are paired up with technical staff to make the transition easier. It is certainly a more daunting task to undertake when faculty learns by trial and error on their own, which may partially explain the high percentage of faculty with no digital content. Additionally, based on open ended answers throughout this survey, it is safe to assume that some faculty are not aware of the resources available to them on campus to receive help with such a transition with some others being convinced that technology does not mesh well with their content. It is also safe to assume that some faculty do not want to devote the time that would be required to learn new skills in this area as long as such endeavors are not rewarded by tenure and promotion processes (Hartman et al., 2007). The good news is that majority of ASU faculty use some type of digital content at some point: 80% use publicly available, free resources, 59% use their own digital content and 65% use online content produced by a publisher.

Chart 22.
Faculty Usse by Types of Digital Content

When asked about their expertise with types of digital content, ASU faculty report highest level of skill and expertise in using publicly available and free resources (60%) where is the chart for this?Zeynokilic 3/13/08 10:00 AM . When asked about their interest in learning about digital content, ASU faculty report moderate to high interest (67-77%) in learning how to produce digital content, or how to locate and use publisher produced content and freely available content (Chart 23). However there is still a sizable group of faculty (23-33%) who has no interest in acquiring skills for creating or finding /using digital content. As technology makes collaboration, production and content sharing easier and more efficient, one might expect the higher education setting to be an excellent place to acquire know-how and experiment with collaborative knowledge production and sharing through innovative curriculum ideas.

Chart 23.
Faculty Interest in learning More about Types of Digital Content

Higher education is an excellent source of collaboration and open exchange of content and courseware (Campbell et al., 2007). As education becomes global and ubiquitous via technology, there are opportunities for team-based course design and delivery (Plater & Chism, 2006). But it may also prove to be a place of resistance for faculty who are accustomed to teaching their own course with complete control and relative privacy. Team approach poses challenges as working in a "high-tech, team-oriented course planning and development process is a new practice for most faculty members" (Xu & Morris, 2007: 37). The relationship with faculty members is a delicate and negotiated role requiring high-level interpersonal skills (Carnevale, 2000; Fredericksen et al., 2000; Luck, 2001). Especially in the case of online course development and technology integration, input from technical personnel and instructional designers might face defensive and resistance behavior from the content-experts (Xu & Morris, 2007).

White (2000) suggests that, regardless of challenges involved, faculty see collaborative development experience as a valuable and inspiring one (also Xu & Morris, 2007; Stark & Lattuca, 1997). Challenges to consider include peer-pressure and concern over quality when a collaboration occurs and when it is publicly disseminated. Faculty satisfaction in a collaborative project is just as important as the student satisfaction with content. For example, in a collaborative e-learning project in the UK, when instructors were not able to alter the course content to fit the local student needs, frustration and resistance built up. Common goals, good coordination and shared responsibility as well as credit given where its due are important for the success of a collaborative effort (Stewart et al.,2006:199; Meyen et al.,1999). Another potential difficulty with team-produced and team-taught content might be managing the legal rights to multiple versions of an object as individuals might adapt it to different purposes. There is also the distinction between copyright ownership, responsibility and distribution rights (Hansen cited in Miller, 2007). Therefore, innovative curriculum initiatives would have to outline and address all these concerns before successfully recruiting faculty to come on board.

Xu & Morris (2007:42-44) found that in developing an online course, faculty were overwhelmingly preoccupied with content development, with little attention given to evaluation and adjustment, instructional process and interactivity, or online learner characteristics. Xu & Morris (2007) also suggest that partnership with someone in the same field but with online teaching experience might be a more successful pairing than a content expert-instructional designer pairing.

Another caveat to consider is, though some of these concepts may be frequently discussed in the educational technology field, some may be completely foreign to faculty in general. And that may be a sizeable proportion of the faculty as well. For example, some ASU faculty struggled with what an expert content curriculum team might mean or what Sakai is (an Open Source Learning Management System, similar to Blackboard and WebCT). We asked ASU faculty to rate their interest in participating in the OpenCourseWare Initiative. The OpenCourseWare (OCW) Consortium is an opensource digital publication of educational materials. These materials are available for free use and adaptation under an open license. The Goals of the Consortium are to extend the reach and impact of opencourseware by encouraging the adoption and adaptation of open educational materials around the world, foster the development of additional opencourseware projects and ensure the long-term sustainability of opencourseware projects by identifying ways to improve effectiveness and reduce costs (www.ocwconsortium.org). In response to the question, 30% of ASU faculty reported no interest in participating in the OpenCourseWare initiative (Chart 24). However, 38% of ASU faculty indicated they were interested or very interested in participating in the initiative with the rest (32%) being at least somewhat interested.

Chart 24.
Faculty Interest in Participating in the OpenCourseWare Initiative

When faculty were asked about their concerns for participating in the OCW initiative they reported around four major themes (Table 7): As discussed in previous sections above, lack of time and incentives are important recurring themes regarding the collaborative model. Some faculty report that they simply do not know enough about the initiative to discuss it. They are unwilling to make a commitment without their roles and responsibilities being clarified in such an undertaking. Others are concerned about copyright and intellectual property issues. Some faculty do not want to relinquish control of their content over to others. Others are skeptical about the compatibility issues with their existing digitized material, as they do not want to invest additional time into reformatting their content again.

Table 7.
Concerns About Participating in the OpnCourseWare Iniative

1) Lack of Time and Recognition

"I am working 200% (really) so my time for doing more has reached the limits."

"Receiving credit for these activities in faculty evaluations."

"Obtaining sufficient recognition for the work in producing such materials."

"Open source is not an employer; there are no system incentives to participate in such ventures, especially at the cost of one's research and yearly evaluations. also, faculty who spend time producing such material are then disadvantaged by other faculty who benefit from this work, but then can invest their time in research instead."

"Considering that I'm evaluated on my research far more than my teaching, I'm interested in time-saving techniques that are applicable to my teaching. But not at the expense of quality of instruction. I think it is important for students to have direct access to their instructor, and when I provide that, the positive feedback I get from the students is overwhelmingly positive. They like seeing humans in front of them."

2) Content Ownership and Control Issues

"Yes, there's a concern that faculty who develop content will not have the rights to their materials, especially if they leave their current institution."

"I am concerned about copyright, the reliability of such materials, and ideological parameters I cannot control."

"Teachers not being allowed to pedagogically adapt and have a certain amount of control over their own curriculum development. "

"Bring it on. Just don't create a policy that forces me to use somebody else's material if it's inferior to my own, ok?"

"Patent right might be infringed."

"My concerns include: Intellectual property being violated and original sources not cited. Research being taken out of context. OCW potentially hurting book sales."

"It would depend on how it was organized and worked. I am not completely comfortable with giving away materials that I have spent years developing on my own with no help, compensation, nor recognition from my parent institution."

"I don't want to share what I've worked hard on with others... and sometimes prefer a bit more privacy about my approach to topics. I also fear we are giving away our knowledge to China and India which will beat us out our own game. Knowledge is power. Why give it away? A bit more realpolitik is needed. Where is Henry Kissinger when you need him?"

"I have concerns participating in anything that is university wide. I do not trust the administration and do not trust that the best of interests of students are being considered over the lining of the pockets of the administration."

3) Lack of information about the initiative unclear about roles and responsibility

"I still do not know what "OCW" has to offer. I have no idea what the materials are to which the above description refers."

"I was looking at the OCW website, and the whole thing seems kind of vague to me. At no point is opencourseware ever defined."

"In general, I am concerned about efforts to make education overly homogeneous, I think that differences in education styles and content is important."

"Who is keeping content up to date? Are there legal problems if one include reference texts from journals and/or books? Who is responsible and what are the responsibilities of everyone involved?"

4) Lack of standardization and compatibility with existing materials

"I am sure there are IP concerns that need to be worked out. I have looked at many of the OCW materials, particularly at MIT, and there seems to be a great disparity, from class to class, with the level of detail in the material as well as the usability of the material. I think that a great deal of work needs to be done in making this material really usable and this would require quite an investment of time and resources. I am not opposed to this, but I feel this is perhaps not well understood by those who make these decisions."

"My principle concern would be how well OpenCourseWare uploads materials from other sources. I have hundreds of test questions and lots of other course material in Blackboard... And I do NOT want to have to reinput all that information into whatever the newest latest thing is."

Responses from ASU faculty, who do not report any concerns about the OCW initiative, can be grouped into three thematic categories (Table 8): Some faculty see technology as the future, not as a choice, but also as a "better" future, for example, with less paper wasted. Others are aware of such initiatives happening around the country and want to be a part of this larger, national/global movement. Faculty in this group are also willing to learn innovative ways to teach. Finally, some faculty indicate that as long as the participation is a choice, and not mandatory, they see no problems with it.

Table 8.
Open-ended answers from ASU faculty who do not have concerns regarding OCW initiative.

Overall, majority of ASU faculty have used digital content at least once. Not surprisingly, more faculty use publicly free digital content than content they themselves created. ASU faculty is evenly split across categories of very interested, somewhat interested and not interested at all in innovative digital curriculum. If ASU philosophy will be moving more towards larger collaborative curriculum projects, more team-based content production and making that content publicly available while using Open Source applications, then we must get the "not interested" faculty to see the value in such engagements. On the other hand, for those who may be interested, issues and concerns such as ownership of content, time and effect on the promotion process as well as how the actual process would work in action need to be addressed clearly.

"Academic review and faculty rewards are increasingly out of sync with new forms of scholarship. The trends toward digital expressions of scholarship and more interdisciplinary and collaborative work continue to move away from the standards of traditional peer-reviewed paper publication. New forms of peer review are emerging, but existing academic practices of specialization and long-honored notions of academic status are persistent barriers to the adoption of new approaches. Given the pace of change, the academy will grow more out of step with how scholarship is actually conducted until constraints imposed by traditional tenure and promotion processes are eased." (New Media Consortium, 2007)

Disaster Planning

Resources

Faculty Survey

The Academic Technology Landscape: Examining Faculty Use and Views at Arizona State University

The Applied Learning Technologies Institute alt^I