Increased interaction of elderly and disabled people with web content after the setting up of World Wide Web (WWW) has created problems such as Web Accessibility.
With the increase in users of Geographical Information System (GIS), it also started facing the problem of web accessibility issues. The users started interacting with the maps extensively in the recent years which ultimately intensified the web accessibility problems in GIS applications. User interaction & content evaluation are helpful in identifying the web accessibility difficulties faced by GIS application users.Web accessibility problems can’t be identified by automated tools or guidelines but users are involved for that purpose. To identify the accessibility problems of users at initial stages, User Centred Approach is used in which users are included in the development process. With the help of user interaction evaluation and content evaluation the author attempts to identify the accessibility issues in GIS applications in this research project. A case study of web based GIS application called MapQuest is undertaken to discover the web accessibility issues pertaining in GIS applications.Another web based GIS application called Google Maps along with MapQuest was evaluated for identifying usability issues.
The usability evaluation of aforementioned two GIS application were conducted using special criterion. Different research methodologies such as usability tests, questionnaires etc were carried out on users from different backgrounds. The report concludes with recommending UCD approach especially in development process through which the web based GIS application’s accessibility can be improved and enhanced.This research project also suggests ways to improve the usability of GIS applications on the basis of studies conducted. TABLE OF CONTENTS ABSTRACTi TABLE OF CONTENTSii CHAPTER 1 INTRODUCTION1 1. 1 Problem statement:1 1.
2 Aims:2 1. 3 Objectives:2 1. 4 Resources:3 1. 5 Risk:3 CHAPTER 2 LITERATURE REVIEW4 2. 1 What is GIS? 4 2. 2 Potential of GIS4 2. 3 Role of GIS/ Web based GIS5 2. 4 Web Accessibility7 2.
5 Importance of Web Accessibility8 2. 6 Web Accessibility Guidelines8 2. 7 Web Accessibility Evaluation9 2. 8 What is User Centred Design (UCD)10 2. Advantages and disadvantages of UCD13 2. 10 Elements of UCD13 2. 11 UCD and Web based GIS14 2.
12 Usability definitions and its role15 CHAPTER 3 RESEARCH METHODOLOGY19 3. 1 Research Process19 3. 2 Literature Review20 3. 3 Experiment21 3. 4 Questionnaires22 3. 5 Interviews23 3.
6 Case study23 CHAPTER 4 EMPIRICAL STUDY25 4. 1 Planning for Web Accessibility Test25 4. 2 Dichotomous method Vs Continuous numerical measurement method25 4. 3 Web Accessibility Guidelines26 4. 4 The need for automatic evaluation27 4. 5 Properties of Good Web Accessibility Metric27 4. Web Accessibility Barrier (WAB)28 4.
7 Construction of plan to evaluate the Web Accessibility of GIS Website – MapQuest29 4. 8 Google Maps & MapQuest usability testing35 4. 9 Questionnaire36 CHAPTER 5 RESULTS / ANALYSIS37 5. 1 First Evaluation- Standard Review Method:37 5.
2 WAB Score of MapQuest:42 5. 3 Second Evaluation – Goal Question Metric Method (GQM)44 5. 4 Over All response of Questionnaire44 5. 5 Role of UCD (User Centred Design) in Accessibility47 5.
5. 1 Interview Results48 5. 6 Section 249 5. 6. 1 Results achieved from usability tests:49 5.
6. List of observation from usability tests51 5. 6. 3 Results from the questionnaires51 5. 6.
4 Comparison of Google & MapQuest53 CHAPTER 6 DISCUSSION/ANALYSIS AND VALIDITY STATEMENTS55 SECTION I55 6. 1 Discussion55 6. 1.
1 Perceivable56 6. 1. 2 Operable57 6. 1.
3 Understandable57 6. 1. 4 Robust58 6.
2 User Centred Design and Accessibility58 6. 2. 1 User Centred Design59 6. 2. 2 User Centred Design and Accessibility Problems59 6. 3 Validity Statement60 6. 3. 1 Credibility60 6.
3. 2 Transferability60 6. 3. 3 Dependability61 6. 3.
4 Confirmability61 6. 4 Section II62 . 4. 1 Effectiveness62 6. 4.
2 Usefulness62 6. 4. 3 User Reaction63 6. 4.
4 Consistency63 6. 4. 5 Architectural and Visual Clarity63 6. 4. 6 Functionality64 6. 5 Validation64 6.
5. 1 Credibility64 6. 5.
2 Transferability65 6. 5. 3 Dependability65 6. 5. 4 Conformability65 6.
6 Selection of Usability Criteria65 CHAPTER 7 CONCLUSION, SUGGESTIONS AND FUTURE WORK67 7. 1 Conclusion67 7. 2 Suggestions68 7. 2. 1 Suggestions for Google Maps69 7. 2.
2 Suggestions for MapQuest70 7. 3 Future Work71 REFERENCES:72 APPENDIX A80 APPENDIX B86 APPENDIX C94 APPENDIX D96 APPENDIX E98APPENDIX F100 APPENDIX G101 CHAPTER 1 INTRODUCTION 1. 1 Problem statement: In the early days of Geographical Information System (GIS), not much emphasize was put upon how users interact with GIS but only on gathering and presenting more and more geographical data. But today, research on geographical data use is focusing more and more on how users interact with this data (Wilson et al, 2008). When GIS became available on the web, its user base expanded dramatically and now it is not only for geographers or others who know geographical terms but also for naive users.With the expansion of the pool of targeted users with variety of skills and knowledge, there is a need of more user-centred design of GIS.
Although great advancements have been made concerning technical aspects and collecting geospatial information, there is a lack of research on human interaction with GIS and behaviour factors (Pick & Gray, 2006). There may be two ways to enhance the accessibility of GIS Applications; either teaches the non expert users about GIS or make GIS usable for non experts through user centred design.The first approach to enhance the accessibility has limitations due to the large and rapidly growing number of non expert users and their lack of understanding of GIS terminologies. Yet another issue is that while making GIS more users centred, the true shape and most of the features of GIS should not be compromised.
Usability is important concerning the web and if any website has problems like file downloading takes time, no clear navigations and not oriented towards the needs of the users, then users get frustrated and they will not use this website in future (Barnum, 2002).The interface of the web application must be so simple that experienced and non experienced users can easily use it and fulfil their requirements with it. The users are only satisfied when they can achieve their goals in a successful and efficient way. (Dillon, 1994) Usability evaluation has methodologies that help to measure usability aspects in system interfaces and also identify the problems.
During the interface design process, Usability evaluation has an important role that includes interactive cycle of designing, prototyping and evaluating (Ivory & Hearst, 2001).Usability evaluation methods play an important role in the designing of most GIS applications because usability aspects included in product design give credibility to company and customers are more satisfied with the products. Map applications are nowadays used by many users who have less experience in using the geospatial data, so usability aspects give benefits to the users accessing internet based services which contain easy to use and attractive exploring functions.Usability thus has an important contribution in the development of successful web-based GIS applications (Nivala et al. , 2008). Koua et al. (2006) proposed usability evaluation criteria for GIS applications; this evaluation helps to assess the ability of GIS applications regarding user performance and satisfaction. This evaluation is very important for evaluation of web-based GIS application because many usability issues can be taken care of through usability testing.
This dissertation will evaluate and discuss the accessibility and usability issues faced by users with different disabilities, different knowledge and different social environments. It will aim to find out how can user centred design (UCD) be helpful to enhance the accessibility of web based GIS? 1. 2 Aims: The goal of this dissertation is to explore issues related to the accessibility of GIS by elderly and disabled people through user-centred approach, and investigate the end-users requirements by conducting usability test, questionnaires and interviews.
1. 3 Objectives: To explore issues related to the accessibility of GIS through user-centred approach * To study the common problems faced by naive users in using GIS applications * To find out the common design attributes for GIS applications that can be accessible for all types of the users * To prove the importance of the end-users in all phases of software development * To study the UI design of GIS and identify shortcomings that can be tuned to increase the accessibility * To identify the role of UCD design in designing GIS applications which are more accessible to users of all abilities * To identify various usability issues. To explore various web based evaluation techniques. To explore such criteria for usability evaluation that can be especially relevant for web-based GIS applications. These criteria could be in relation to the specific challenges that these applications have, and might be somewhat different than traditional usability criteria, due to the fact that usability evaluation was developed earlier than web-based GIS applications. 1. 4 Resources: The resources needed to complete this dissertation successfully will be Library Services, Internet.Library service will provide access to books, journals, articles to conduct the research required to complete the dissertation.
Internet services will also be used extensively for research and also it will be useful for sending questionnaires to the volunteers, and analysing the case studies that will be used in this dissertation. Also ‘EvalAccess 2. 0’ tool which is software and will be used to compare the accessibility of GIS applications. 1. 5 Risk: This dissertation will focus on accessibility and usability issues of disabled people as a whole. Because of time constraint it doesn’t choose limited users from each category of disability.Choosing each category of disability would bring up more concise results.
CHAPTER 2 LITERATURE REVIEW This chapter is organized into three sections. Section 1 discusses the role of Web GIS, the potential of Web based GIS and who are its users. Further, HCI related issues in web based GIS are discussed. Section 2 discusses the evolution of Web Accessibility and the importance of accessibility of web contents.
It also researches the guidelines and web standards regarding accessibility made by the international community World Wide Web consortium (W3C).Then the accessibility of web based GIS. In section 3 the overlook of what is User Centred Design, its elements and relevance of User Centred Design and Accessibility is presented. 2. 1 What is GIS? Good science is said to begin with clear definitions; but it is hard to give a clear cut definition of geographical information systems. As a result, different definitions have been proposed with the passage of time depending on the context.
So we can define the term geographical information system in a variety of ways and each definition depends upon what we are searching for.The immense in GIS in recent years is due to the advancement in other technologies and fields that correlate and support the GIS system, such as geography and cartography, geo-visualization, computer science, database theory and mathematics field. Ron Abler has defined GIS as “Not just one but many simultaneous technological revolutions” (Clarke, 2004). So it is clear that GIS is not a new killer-application but it is an essential and innovative application like spreadsheet, database management systems and word processors.Another author, Peter Burrough, has defined GIS as “A powerful set of tools for storing and retrieving at will, transforming and displaying spatial data from the real world for a particular set of purposes”.
Different authors have tried to define GIS in terms of its functionality. So another definition of GIS is “An automated system for the capture, storage, retrieval, analysis and display of spatial data”. (Clarke, 2004) 2.
2 Potential of GIS From the above it can be concluded that geographical information systems are known as powerful and integrating tools for managing and analyzing geo spatial data.The merging of the GIS technology with internet has introduced a new area of research referred to as web based GIS, on-line GIS, internet GIS and internet distributed GIS services. As accessibility to the web is easier for every one; web based GIS plays a major role by offering GIS functionality on the web (Kraak, 2004). Initially static maps have been developed then interactive maps with zooming features were introduced and now dynamic maps have been introduced with multiple features to give maximum functionality and accessibility to its users as a virtual communication tool.Web based GIS is still an emerging field and it is a broad area which involves a lot of issues that still need to be researched, such as spatial analysis and modelling, spatial database designing, wireless and mobile services and 3D data access and query. (Dragicevic, 2004) The World Wide Web has also altered the role of maps and makes them more dynamic. The map can be an essential part of the search engine, especially if we are referring to geospatial data. Maps can also play an important role by acting as an interface for geographical and non geographical information on the net.
Maps can be helpful in guiding us from location ‘A’ to ‘B’, viewing the arrangement of the landscape, displaying urban plans and show changes occurring in population distributions. Hardware and software developments have made a tremendous change in the scientific and social needs for spatial data and for maps. The web not only presents dynamic data but also gives the user the ability to interact with the system, and there is subsequently a growing demand of the current users for faster and real time access to data. The role of the maps is definitely central in the web based GIS environment.The dynamic and interactive nature of the maps assists the users in solving geo-spatial analysis problems.
(Kraak, 2004) 2. 3 Role of GIS/ Web based GIS The role of GIS is broad and it can act as an information system, support for decision making and may be helpful for emergency management, crises management and earth quake situations. Clarke (2004) quotes that Jack Estes and the late Jeffrey Star has taken GIS as an information system and they defined this as “An information system that is designed to work with data referenced by spatial or geo-graphic coordinates.In other words, a GIS is both a database system with specific capabilities for spatial-referenced data, as well as a set of operations for working with the data”. Based upon the above definition we can say that GIS is an information system that answers the user‘s queries. Whenever it receives a query from the user side, it collects relevant data, filters and sorts that data and then displays it before the user in a relevant form. Thus we can say that a GIS tool gives the solution of generic problems.GIS has proven its importance worldwide and web based GIS technology has shown its advantages in a variety of areas such as planning applications, decision making, management of historical buildings, and e-government etc.
Web based GIS fulfils the requirements of e-government by sharing geographical data both within the organization and with the outside world. Geography is in some sense the glue that can integrate the data scattered in different departments and lying at different locations. Web based GIS is the first step in e-government that ensures the sharing of essential information between all the departments in local authority.Thus this technology enhances the internal and external communication, operations, efficiency within the organization and builds up a positive relationship with its citizens and/or visitors.
Figure 2. 1 GIS working as glue, integrate scattered data (Stachowicz, 2004) Web based GIS provides centralized and easy access to spatial data for the users. It is the demand of the users for up-to-date, timely and rapid access to the data. Web based GIS fulfil all these requirements and provide dynamic geographical information to the users in an easy and accessible format.
Web based maps assist the users in location finding, route finding and quick decision making. 2. 4 Web Accessibility When we talk about accessibility of a web page, it means that the web page should be accessible to almost every one with all abilities. During the start of World Wide Web, pages were more accessible because almost all web pages consisted of text which could be made more accessible with less complexity. With the advancement of web technologies and hence addition of more complexity, it is now difficult to make web pages accessible to all.
Web designers started to include images, frames, streaming audio and video and different kinds of applications, which have made web pages full of hurdles for users with disabilities. (Hackett, 2004) The awareness is on rise about user‘s rights about the access of World Wide Web and there is recognition worldwide about the same rights of access to information on Web for people with disabilities. Legislation about the accessibility of web contents in different countries portrays the importance of the issue of making information on web accessible for all. Richards, 2004) Most of the developed countries have built their own guidelines or refined WCAG guidelines to make web contents more accessible. Increasing accessibility of web content is not only beneficial for people with disabilities but also for common users with lower ebb of resources like lower connection speed, lower processing speed. Mynatt (2000) believes as the average age of world population is increasing as a result there will be increase in the group of senior citizens accessing information on the web.With aging, most people have deprived some of their abilities like focus, resolve images and distinguish colours. So if we look at cost to benefits ratio, it is clearly indicating more benefits than cost.
It may increase the cost one to percent to make website accessible but it will result in increase of users by almost twenty percent. Web Accessibility is directed to address different kind of users and accessibility varies depending upon the disability type of the users.The users which could be addressed by Web Accessibility may have the following disabilities * Visual: From intense degree of blindness to low vision and colour blindness all include in this group * Mobility: People feeling difficulty in using input devices particularly mouse due to immobility or loss of control of relevant muscle * Auditory: people feeling difficulty or disability to listen * Cognitive: peoples having difficulty or disability to understand, learn, memorize or giving attention etc. 2.
5 Importance of Web AccessibilityThere is general perception that accessibility means web contents should be make accessible to users with disabilities like blind users. The area of web accessibility deals more than this limited view. In broader view Web Accessibility means making web contents available for as many users as possible. The user may range from common user to user with disabilities such as vision, mobility or learning impairment, users having limited network speed, slow processing speed compelling browsers to show text only version of websites and also users with mobile phone and PDAs with limited bandwidth and memory.
Jackson, 2003) If we look at the cost benefit equation of making web content accessible we can clearly find this equation giving more benefits than cost by increasing users with substantial effort. For example minor changing of adding ALT tags for images will make large some difference. Advantages of making web contents more accessible can be increased by broadening the definition of ‘who’ will get gain more access by this adaptation. Elder people are big percent of whole population and if you ignore web accessibility you are simply ignoring large chunk of possible users.Also this portion of population posses more free time and also discretionary spending power.
Web contents can be converted into more accessible for adults by making the contents easy to read, reducing cognition and more efficient and easy use of keyboard and mouse. (Richards, 2004) 2. 6 Web Accessibility Guidelines In 1997 the Web Accessibility Initiative (WAI) was setup by World Wide Web Consortium (W3C) to define international standards for users with disabilities. The standards are divided into three different groups.
. Web Content Accessibility Guidelines 2. User Agent Accessibility Guidelines 3. Authoring Tools Accessibility Guidelines WCAG 1.
0 the first version of web content guidelines is mainly for websites with static information. But with advancement in web technology these guidelines are not sufficient to handle the emerging issues related to accessibility of web contents. To meet this challenging situation W3C‘s Web Accessibility Initiative gave a new version of guidelines namely WCAG 2. 0 (Reid & Snow-Weaver, 2008).Since 1997 W3C‘s Web Accessible Initiative is successfully promoting WCAG around the world. Many organizations have taken benefits from these guidelines in changing their web contents into more accessible form. Also there is more inclination now at national level to adopt these guidelines to standardize the web content.
(Kelly et al, 2007) 2. 7 Web Accessibility Evaluation Accessibility of web content is evaluated to check the degree of accessibility available to the contents for users of all kinds. There are different methods to evaluate the web accessibility of any web content.Generally web content can be considered accessible if it follows the regulation provided by WAI‘s WCAG 2. 0 (Villegas et al, 2009). The guidelines provided by WCAG 2. 0 are divided into different set of check points and every checkpoint is assigned different level according to severity (Freire et al, 2008).
One of the web accessibility evaluation methods is to involve user in this evaluation process. The user will interact and explore the real problems relating to accessibility. Strictly adopting the guidelines provided by WAI‘s WCAG does not guarantee the success.User’s experience is necessary to validate the accessibility of web content. There are some automated tools to evaluate the accessibility of any web content. According to WAI the following documents can be part of evaluation of web accessibility. a) Preliminary Review of web content for Accessibility b) Conformance evaluation of web content for Accessibility c) Involving Users in Web Accessibility Evaluation d) Evaluation of Web content Accessibility through tools. So evaluation of accessibility of any web content depends upon adherence of guidelines and user satisfaction level.
In this thesis project we have evaluate accessibility of GIS web content by firstly using tool to check the adherence of guidelines provided by WAI‘s WCAG 2. 0 and then conduct survey of users with disabilities. 2. 8 What is User Centred Design (UCD) UCD is a broad term and it is defined as the design philosophy or process in which needs, limitations and demands of end users of an interface have been extensively considered in the entire design process. The entire design process revolves around the end users who have core role in the design process.In other words, UCD is a multi-stage problem solving process in which designer not only analyzes how the users interact with the interface but also perform a test to validate their assumptions with respect to user behaviour in real world scenario. The main distinction of UCD from other design philosophies is that UCD optimizes the user interface according to which users feel easiness and want to work rather than bounding the user to change their behaviour according to the design approach.
Currently, different UCD models and approaches have been used to design a product and each approach has its own pros and cons.Models assist the software designers in engineering a product for their users. In these models requirement of the users are kept under consideration right from the beginning till to the end of the product cycle.
All the following models follow ISO (ISO 13407 model, 1999) standard. These models are as: Cooperative design: This approach evolved in Scandinavia since 1970. End-users are involved very early in the design process. Thus end-user is given a ‘voice’ in the design and development process and ultimately this thing enhances the quality of end results.Cooperative design demands the right set of participants, right location and physical space. It is also essential to create such environment in which all participants can contribute actively to get some meaningful results. (Bodker et al, 2000) Due to cultural difference among users and designers it may be possible that users are unable to understand the language of designers. So for this purpose it is recommended that prototypes such as mock-ups (three dimensional paper based representation) or a paper based outline for web screen shots or other product.
Abras et al, 2004) Participatory design: It correlates with the cooperative design in which end users, developers and business representatives work collaboratively. This design model is more beneficial in early stages of design process. This design model gives a lot of advantages such as * Provide equal participation opportunity to technical and non-technical members * Provide the close collaboration between developers and end users * Provide a forum for investigating product related issues.
(Gaffney, 1999)Contextual design: Context design also called customer centre design provides the assistance to cross-functional team to reach an agreement on what the customer‘s needs and how to design the product for their customers. Cross-functional team listen the whole story of an interview but confine the concepts and issues related to design problem. Contextual design user says “When I was coding I was behind a mirror.
.. but when I sat together with the user in front of the system, I felt like I was looking through the mirror and becoming aware that there was a human being on the other side”. Holtzblatt, 2001) The term ‘User Centred Design’ was presented by the Donald Norman in 1980s and then this term widely spread after the publication of the book entitled: User-Centred System Design: New Perspectives on Human-Computer Interaction (Norman & Draper, 1986). In the book The Psychology of Everyday Things (POET) (Norman, 1988), Norman has further built on UCD concepts. He has proposed four suggestions related to interface design * The interface should be simple enough in order to determine the possible actions at any moment. The things on the interface should be visible.
* Current state of the system should be evaluated easily. * Interface provides the natural mappings between the required actions and intentions; between actions and the resulting effect; and between the interpretation of the system state and visible information. According to Donald Norman recommendation, user should be in the centre of the design. Designer should design the interface that assists the users in performing their tasks and users have to put a little effort to learn a system.Small pamphlet should be delivered along with the products but that pamphlets should be understandable. Just to say designer the design of the product should be intuitive is not enough.
There are some designs principles that should be adopted to give better outcome. Norman (1988) has recommended seven designs principles that can assist the designers in their task. 1. Utilize the knowledge both in the world and in the head. Build the manual before implementing the design and it should be in understandable format. 2.
Structure of the tasks should be simple that giving full control to the user ver tasks. Tasks provide mental aids to the user for easy archival of information. 3. Things should be visible such that if user presses right click (mouse) on an object it exposes its full functionality for the user. 4. Graphics or icons should be used to make the things understandable.
5. Constraints both artificial and natural should be used to restrict the users. 6. After proper error tracking, the error should be displayed before the user in readable and understandable format so that user can perform error recovery. 7.
Standardize the action if all else fails to avoid any abnormal action.In short all the work performed by Norman stressed on the exploration of user needs and requirements before designing the product. 2. 9 Advantages and disadvantages of UCD Table 2. 1: Advantages/disadvantages of UCD Advantages Disadvantages 2. 10 Elements of UCD According to Katz-Haas (2004), UCD model have the following four elements.
1. Visibility: Visibility provides the assistance to the user in constructing the mental model about the things. Essential elements such as those that aids in navigation should be highly visible for the user.Visibility is such that user can predict what he can do and what he cannot at a glance. The important information should be made prominent and placed at top place. 2. Accessibility: Accessibility is another element of UCD in which information is made available for the users easily and quickly no matter what the size of the document either it is long or short. Avoid the users from brainstorming to find specific information.
Variety of ways have been provided to look into the information such as search function, page no and navigation elements. Chunking’ technique should be used in which information is divided into sections or blocks and relevant information is kept together and organized in a meaningful manner. Bold and italic pattern is used to make the information prominent. 3. Legibility: Text such as text size, font and style should be used that make the information convenient for the readers.
It is hard to read such text or information written in ornamental fonts or all in capital letters but making bold and italic to the text can be helpful for the readers. High-figure-ground contrast between text and background can enhance the legibility such as dark text on light ackground. 4. Language: Information should be displayed in short sentences and in simple words instead of using technical terms. Utilize the active voice or verbs for writing information instead of noun strings. 2. 11 UCD and Web based GIS Although GIS are dominating in the market and usage of them are increasing but on the side usability of GIS technology is pitiable. Poorly designed GIS user interface can be made usable by arranging the training for them but this solution is not good according to the economic point of view.
Relating to GIS user interface it is said “GIS user interfaces should be designed such that they fulfil user requirements instead of adapting the end-user to a GIS”. Comprehensive and accurate knowledge of the user requirements acts as a prerequisite for the GIS customization and development and it is said “User requirements are the criteria against which quality of GIS is tested”. Quality of use is also defined and measured like other technical and economical quality aspects. Quality of use is a multi-dimensional concept in which multiple aspects are checked such as robustness, visibility; learn ability, legibility and accessibility.Quality of use should be evaluated during the customization and development phase in order to get rid of the expensive market failures later. The development of GIS is different from other software development processes as GIS user interfaces are very complex and to make them usable it is essential to tailor them to specific user requirements. Mostly users interact with web based GI data and modify the parameters to achieve their desired results on the screen so it is crucial to provide interactive interface between end-users and the GIS.
The design principles of UCD may be helpful for creating effective, efficient, innovative and interactive GIS user interfaces. Focus on end-users Iterative design appropriate allocation of functions between end-user and GIS Multidisciplinary design team (European Commission, 2005) 1. Focus on end-users End-users can contribute a lot in the design process of GI user interface by sharing knowledge and experience about GIS domain, tasks, workflows, objective, responsibilities and working environment. Users participate actively or passively in this development process.Users just state the needs, demands and requirements in passive participation but in active participation users have close collaboration with the designer or sometimes users itself design the interface. Representative users having different hand-in experience and working in different environments should be involved to analyze variety of requirements. 2. Iterative design Iterative design cycle can enhance the quality in use and shape the interface according to user demands.
Two to three design cycles are enough in GIS user interface development process.Iterative design process can minimize the designing errors and reduces the chance of failure of application. 3. Appropriate allocation of functions between end-user and GIS Proper functions should be allocated between GI and users. Tasks should not be completed in single function rather to use procedure (sequence of GIS functions) to process the task.
It is also beneficial for end-users if functions are hidden behind macros for task execution. 4. Multidisciplinary design team Variety of skills is required to design GIS user interface such as domain experts, technology experts and design experts.So users from different groups should be involved but this multi-disciplinary team should be limited in size. 2. 12 Usability definitions and its role In the field of software development process, Usability plays quite an important role. It can be defined in many ways like the other software engineering terms (Folmer & Bosch 2004).
According to the International standard organization (ISO 9241-11) usability can be defined as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (Jokela et al. 003). The importance of usability engineering was recognised by Shackel & Richardson (1991) and had also proposed to measure usability by four different criteria namely effectiveness, learnability, flexibility and attitude; where in Effectiveness deals with the performance in accomplishment of tasks, learnability is degree of learning for achieving tasks, flexibility is related to the ability of adapting to the variation of tasks and attitude relates to the users satisfy with the system (Folmer & Bosch 2004).A detailed definition of Usability was given by Nielsen (1993), where he associated it with five important attributes and these attributes are learnability, efficiency, memorability, errors and satisfaction. A concise explanation of the five most important attributes of usability were defined by Nielsen as they have a key role for the user’s perspective and perception of most interfaces.
Learnability: The system must be so simple and easy to learn that users that no difficulty is faced by the users and can be learned quickly.Efficiency: The system should be efficient and not time consuming for users when they work with it. Memorability: The system should be easy to memorize by the users so that when they use this system after some time, they should be able to recollect as to how to go about. Errors: The system must have low error rate and should help the users to avoid making many errors while using it. Satisfaction: The system should be pleasant to work with which assures that the users are likely to use it again and are satisfied with the system.Nielsen presented how these different criteria operate which show a clear concept of usability and its role in the larger issue of acceptability. Acceptability can be comprised into practical and social acceptability. Usability and utility can help in product usefulness that makes it easier for the user to carry out a task.
According to Nielsen, usability influences product acceptability among users. For last 15 years, ISO organization has played an important role in the development of many Human Computer Interaction (HCI) and usability standards which are used in the industry to impose consistency.It has also developed standards for its components like icons, cursors controls etc. The three attributes of usability presented by the ISO 9241-11 are described as follows: Effectiveness: The system is required to be accurate and have complete functions such that users have no difficulty in achieving their goals with its accuracy and completeness. Efficiency: The resources of the system must be used with accuracy and completeness through which users achieve their goals.
Satisfaction: The users must be contented after using the system. From the user? perspective, this definition holds two different views which are user performance view that includes effectiveness and efficiency and the other is the user view which is concerned with the issue of satisfaction. The ISO 9241-11standard is used to identify the important information which must be considered for evaluating usability of user performance and satisfaction (Folmer & Bosch 2004). Many of the products used as part of everyday life having no usability and not fulfilling the users’ requirements leads to frustrating them, hence they choose to not use them again (Jordan, 1998).Usability is an important aspect concerning web applications, as is obvious from the statement of Jacob Nielsen “Usability rules the web, simply stated that if the users have difficulty in accessing the website contents, they will not use it”.
Those Websites that have poor usability usually face problems like downloading file which is time consuming, navigation not clear, not oriented towards the needs of users, etc. Such problems repeatedly faced by users would compel them and the others not to use it (Barnum 2002; Nielsen 2000).Any website or interface should be designed in a way that both expert users and non experienced users are able to use it without any difficulty. The success of any website or interface depends upon the user? s satisfaction which would make their work done in an easy and efficient way. In addition to the attraction of websites regarding using well thought through colour schemes, web sites also need to help the users to accomplish what they want to use them for.Eventhoug a website performs well due to its functionalities, but users are unable to understand how to go about it, then it does not prove to be a usable website as it does not fulfil their requirements (Dillon 1994).
Only when important usability evaluation criteria are taken into consideration during the design and development process, the quality of product or interface can be successful (Stone, 2001). Koua et al. (2006) propose usability evaluation criteria for GIS applications for user-based and task-based evaluation.They argue that this evaluation intends to assess GIS application? s ability to meet user performance and satisfaction. Moreover their assessment focuses on the effectiveness, usefulness and performance of the application which is important for such applications since usability testing is based on these criteria thus provides good understanding of usability issues in web-based GIS applications. Their proposed assessment methodology includes three criteria which are: effectiveness/user performance, usefulness and user reaction.According to Koua et al.
(2006) effectiveness deals with application functionality and effectiveness evaluation is based on observations of user performances and experiences of the tasks. It also deals with the gathering of data and any parameters available to complete a task which is measured by the time that is spent on to complete it, number of complete and incomplete, errors and error rate, whether carried out correctly or not and their performance and response, level of difficulty and time spent for getting help.Usefulness has to do with the user’s expectations and demands while carrying out different tasks that depend on whether the application is supportive of user? s goals and tasks, if users can easily understand and interpret the application results, whether it’s flexible according to user expectations. Through user’s comments, task performance, verbal protocols and responses to questionnaires, the data for the above can be collected (Koua et al. 2006). User reaction deals with user’s opinions, views, attitudes and user preferences towards application which can be measured by questionnaires, rating and interviews as well (Koua et al. 006).
Information Services and Technology (IS;T) at Massachusetts Institute of Technology (MIT) is a group working on usability aspects of websites providing the services like, group reviews of the websites (heuristics reviews), accessibility testing, usability testing, Card sorting, Pluralistic walkthroughs and Consultation for usability tests of products (Usability @ MIT, 2009). They also work continuously for developing guidelines for usability evaluation of websites.According to them most factors that could be evolved in usability evaluation of any web sites concern the following (but it should be kept in mind that it is not necessary to apply all factors to all web sites) navigation, functionality, user control, language and content, online help and user guides, system and user feedback, web accessibility, consistency, error prevention and correction, and architectural and visual clarity (Usability Guidelines, 2009). The criteria proposed by Koua et al. (2006) have been adopted for our usability evaluation of the system and is ideally developed for GIS applications.Furthermore, the authors also adopted IS;T guidelines along with selected criteria of Koua et al. (2006) for designing of questionnaire.
The selection of Koua et al. (2006) covers the specific usability evaluation of GIS application and IS;T covers the broader web aspects of our selected web-based GIS applications since the main focus is on GIS application’s usability evaluation. CHAPTER 3 RESEARCH METHODOLOGY In this chapter, the chosen research methodology is discussed. This section explains the research approaches, methods or techniques that author have chosen for this study and why they have been chosen. . 1 Research Process Research is defined as the systematic way to investigate the solution of problems in research work.
In this case author wanted to conduct the research on ‘Geographical information system (GIS)’. After studying the literature related to GIS, author identified certain gaps in the research and tried to fill one of these gaps. After brainstorming, author formulated the topic “Investigating usability and accessibility issues pertaining to Web Based GIS Application for elderly and disabled people”.Under this topic a number of research questions were posed which the author have tried to answer and for this purpose different research methods and approaches were used that are further discussed in this section one by one.
Each approach provided a variety of data in order to find the answers to the research questions and to validate the results. The research process that has been conducted in the research work is shown in fig 3. 1 in order to promote a better understanding of the overall research process. Figure 3. 1: Research Process (Blaxter, 2001) 3. 2 Literature ReviewAuthor planned to conduct a literature survey to hold on this study as this is the best tool particularly in this area of research. Researchers also call this process ‘doing a survey’, ‘literature search’, or ‘study’ or ‘doing a literature review’.
(Verma, 1998) “Literature refers to all available research on this subject and ‘Literature Search’ refers to the process of finding the material and a ‘literature survey’ simply describes the literature which exists. ” (Blaxter, 2001) Literature survey methodology is a cheap and effective way of gathering data particularly in short period of time.The end results may be analyzed rapidly as this is also the goal of this study. This methodology produces suitable and consistent data which can be generalized with more assurance. Literature survey may be comparatively easy to manage and does not need any fieldwork. (Verma, 1998) Author have aimed to carry out a comprehensive literature review to discover what has already been done in this field, gather data according to the angle and then execute analysis on that data. Through the literature review, author has aimed to accomplish high quality of data (Blaxter, 2001). In this research area, literature survey was used as an iterative cycle.
In the first cycle we have gathered data, analyzed that data and if we perceive some deficiency in that data then second iteration of this cycle has been performed and in the same way iterations have been executed until satisfactory answers were found for the research questions. This iterative process is shown in the figure 3. 2. Figure 3. 2: Iterative Cycle (Blaxter, 2001) The main objective of the literature review is to learn from previous work and provide a foundation for the research work based on the brainstorming of the previous researchers in this field (Dawson, 2005).For literature review different sources have been used as shown in figure 3. 3. Figure 3.
3: Sources of literature review (Dawson, 2005) 3. 3 Experiment After the literature review author came to the point where a test was conducted to evaluate the web accessibility of MapQuest. For this purpose it was decided to use the automated tool EvalAccess 2. 0 in this report.
But this tool tests the accessibility based on some guidelines and it is a content based evaluation. So this evaluation is not sufficient to check the accessibility.Therefore Questionnaire technique is used to further evaluate the accessibility of MapQuest and this is a user interaction based evaluation. 3. 4 Questionnaires Questionnaire is an excellent technique that is used to accumulate demographic data and user views related to a topic (Preece, 2002). The Questionnaire should be designed in a careful manner because confusing and redundant questions can lead to the breakdown of this process and wastage of resources. Questions should be clear and close ended so that users feel easiness in answering those questions (Preece, 2002).
Questionnaire technique is used as an important tool to gather data from users and it is also one of the cheapest tools for evaluation of a system. Author has used this technique to evaluate the first research question. The questionnaire was designed on the basis of data retrieved from the literature review and experiments. To avoid ethical issues in the limited study author chooses selected elderly people who may have some type of vision and hand mobility problems due to old age. This process is shown in the figure 3. 4. Each user is given a task to perform on MapQuest web application.
After this, a questionnaire, which is designed relevant to the accessibility issues of the web based GIS applications (MapQuest), is given to each user and feedback is collected from them. Figure 3. 4: Questionnaires process (Preece, 2002) 3. 5 Interviews The interview is a type of query and it is defined as “a conversation with a purpose” by Kahn and Canell (Preece, 2002). Interviews can be conducted as ordinary conversation related to some specific topic or questions related to the topic can be asked from the users to get their comments, observations and experience about that topic.By conducting interviews with users, quantitative and/or qualitative data is collected from the participants to evaluate the target (Preece, 2002). There are many types of interviews such as unstructured interview, structured interview and semi-structured interview. Each type has its own significance and choice of type depends upon the situation in which it is being used.
Structured interview is predetermined and it is more suitable when we have a clear idea about the goal and we ask related questions from the users. Author has used structured interview approach to evaluate the second research question in this thesis.For this purpose, author planned to visit a company and conduct an open ended conversation like interview with the head of the development team that is working on web based GIS applications.
The plan was to first get some briefing about the development process and accessibility issues in the GIS applications developed at the company. To get better results from the interviews these guidelines may be helpful. It is recommended that you avoid asking difficult, compound and irritating questions from the users.
Questions should be simple, clear and understandable for the users.Be attentive during the interview. Ask the question in a natural way and encourage the user response and do not give your own opinion (liking and disliking) on user response.
(Nielsen, 1993) 3. 6 Case study Case study is a qualitative approach with the focus of in-depth study of a particular field. In the beginning we tried to identify accessibility issues generally faced by users in almost every web based GIS application. At the end author has conducted a case study in our research work with the ambition to evaluate the accessibility issues particularly related to MapQuest application.For this purpose author has aimed to match MapQuest with general accessibility issues and perceive to what extent MapQuest fulfils these requirements. In the next step it is compared with different GIS applications from accessibility perspective. Related material search on the internet and user observations and comments have been included in this work.
With the rapid evolution of World Wide Web different online applications became available to common users and web based GIS is one of them. MapQuest is used as a case study application in this thesis project.MapQuest is the oldest web based GIS application since it was founded as a cartographic service in 1969. MapQuest started as internet service in 1996. It was the most popular and most widely used web based GIS application before the advent of Google Maps.
Still it is a widely used web based GIS application in the world. MapQuest claims distributing over 20 million maps on daily bases. (Peterson, 2001) CHAPTER 4 EMPIRICAL STUDY This chapter introduces the plan overview for evaluating the accessibility of MapQuest.Different terms and methodologies have been used in the conduction of the experiment so a comprehensive discussion of each of them is also given in this chapter in order to create better understanding of the test plan. 4. 1 Planning for Web Accessibility Test This dissertation uses quantitative approach to conduct the web accessibility evaluation. To quantify the attributes of known objects is vital in enhancing the state of science of any field. Currently web, one of the most growing and researching object has generated many metrics that are helpful in scientific investigation (Dhyani et al, 2002).
To measure web accessibility in accurate and quantitative form is very critical due to many reasons. First it will enhance our understanding related to the web. Secondly we can determine the current situation of web accessibility. Compare and rank different sites based on the accessibility and even the accessibility level of single site can be judged at different times. All electronic information should be accessible to all the persons in the world and websites are the significant spots that represent the information before users and people rely on that for electronic information.Persons with different disabilities also access them for electronic information, so websites should be designed in such a way that people with disabilities can interact and access them easily. Before explaining the plan how the author will conduct the web accessibility evaluation test, it will be better to discuss some ideas, terms and techniques.
The author have planned and structured the accessibility test on same outline. A comprehensive discussion on the following headings will assist in understanding our test plan. 4. 2 Dichotomous method Vs Continuous numerical measurement method There are two ways to measure the web accessibility.One is more rigid called dichotomous method in which it is checked either site is accessible or not accessible. The result of dichotomous method is absolute compliance with the standard guideline called web content accessibility guideline (WCAG).
But this method is more rigid and may lead to inaccuracy in accessibility labelling. The other method, continuous numerical measurement method determines the accessibility in term of ‘degree’ rather than dichotomous (accessible-inaccessible). It categorise the websites into four accessibility degrees having name 1) highly accessible, 2) mostly accessible, 3) partly accessible and ) inaccessible This method is more flexible that offer discrimination power and support scientific approach (Parmanto ; Zeng, 2005). 4. 3 Web Accessibility Guidelines Abundant guidelines have been developed by different technical and skilled persons to make the website accessible specially to disable persons. According to Paciello (2005), the first initiative towards web design standards was taken by city of San Jose, California to make the web accessible for disabled persons.
In 1997, Web Accessibility Initiative (WAI) was established by W3C which proposed the Web Content Accessibility Guideline version 1. (WCAG) in 1997 (Ray ; Ray, 1998; WAI, 2010). The latest version WCAG 2. 0 has been published in 2008 which is capable of evaluating the multi-media websites more precisely. Tufte (1997) points out that WCAG is assumed as stable and international specification that is used as a base to evaluate the accessibility of web contents with disabled persons. WCAG is organized into 14 guidelines that are further divided into 91 checkpoints.
These checkpoints are used to evaluate web accessibility and they are further divided into three priority levels (Parmanto ; Zeng, 2005). * Priority 1It contains 29 checkpoints. For example provide text to non-text element that serves the same purpose, present the steps in a sequence if a task consists of a no. of steps, information should be operable through keyboard, all these checkpoints belong to priority l. These checkpoints must be fulfilled during the web development otherwise it will leave a hole in web accessibility. * Priority 2 It contains 40 checkpoints such as headings and labels clearly express the topic or purpose, suggestions should be presented in front of the user against an error, all these checkpoints relate to priority 2.These checkpoints may be fulfilled to make the site accessible to the users having any kind of disability. The violations of these checkpoints may degrade the web accessibility.
* Priority 3 It contains 22 checkpoints such as user can continue his/her activity or task after re-authentication whenever an authenticated session expires, context-sensitive help should be available, all these checkpoints are included in priority 3. These checkpoints may be fulfilled if we want to make the site maximum accessible for all types of users. Parmanto, 2005) WCAG conformance logos have been introduced by WAI which are used by different companies/organizations on their sites to claim conformance level. This thing will create awareness of accessibility issues in the public. The conformance levels are defined as * Conformance level ‘A’: This level meets all priority 1 checkpoints.
* Conformance level ‘AA’: This level meets all priority 1 and 2 checkpoints. * Conformance level ‘AAA’: This level meets all priority 1, 2 and 3 checkpoints. (Parmanto, 2005) 4. 4 The need for automatic evaluationDue to rapid development in web technology volume of web pages has also increased. In this scenario automatic scoring and evaluation approach is more preferable than manual. Some other advantages of automatic evaluations are cost reduction, minimize effort and time to conduct the evaluation, increased consistency in final results, reduce the demand for accessibility expertise and finally assist in web design process. (Parmanto, 2005) 4. 5 Properties of Good Web Accessibility Metric Metrics are helpful in controlling, understanding and improving a product.
In accessibility context metrics are more beneficial in improving the accessibility of target products and companies may introduce accessibility issues in the development process from the beginning of the process. Many researchers have discovered variety of accessibility metrics for web based systems and goal of those metrics is to summarize the results in quantitative form (Parmanto, 2005). Researchers have proposed the following properties of good web accessibility metric. (Tufte, 1997) * Metrics should measure the accessibility in numeric form * Metrics should measure the value in continuous range instead of dichotomous *Metrics should be fair that also consider size and complexity of the site.
* Metrics should be scalable 4. 6 Web Accessibility Barrier (WAB) WAB method compares the web contents against standard checkpoints defined in WCAG and then determines the number of violations against each checkpoint. It is the component that determines the score called Web accessibility barrier (WAB) score. If a website has fewer accessibility checkpoint violations then it is assumed as having fewer barriers for disabled persons and the WAB score of the site will also be lower.As previously discussed WCAG divides the checkpoints into three priority levels but in WAB scoring these priority levels are used in the reverse order.
For example, in priority 1 the weight age of WAB score is three times greater than priority 3 as persons with disabilities feel more web accessibility problems in priority 1 violations. It means that lower WAB score indicates lower accessibility barriers and higher WAB score denotes higher accessibility barriers for the persons with disabilities.If a website having zero WAB score then it means it has no accessibility barriers and fully accessible to the people with disabilities.
(Parmanto, 2005) However, WAB metric is not fair in the sense that a web page with 5 violations (? Image without alternative text‘) out of 500 image objects and another web page with 1 violation out of 1 image object endorsed in that page. So “the number of actual violations of a checkpoint must be normalized against the number of potential violations of a checkpoint” (Paciello, 2000).For example ‘Image without alternative text’ is a true violation and potential violation consists of all image objects over a page. (Parmanto, 2005) Final WAB score is calculated as the average score of all the pages within a site. Mathematically, it is written as Fig 4. 1: WAB formula (Parmanto, 2005) 4. 7 Construction of plan to evaluate the Web Accessibility of GIS Website – MapQuest MapQuest is the oldest web based GIS application since it was founded as cartographic service in 1969. It was most popular and most widely used web based GIS application before the advent of Google Maps.Still it is widely used as web based GIS application in the world. We have performed the accessibility evaluation test by focusing on the web based GIS application MapQuest that we have also chosen as our case study. We have aimed to find out accessibility issues facing the users with disabilities in accessing MapQuest. Different methods and approaches are in common in evaluating the accessibility of web based applications. But we have planned to use the following methods to evaluate the accessibility of MapQuest. Fig 4. 2 General view of MapQuest (MapQuest, 2010) Standards Review methodStandards review is a user-centred process that determines either a product conforms to the interface design standards. Different software or tools are available in the market to check the accessibility of web pages in a site. To perform the accessibility test with MapQuest we have followed internationally accepted standards defined in WCAG1. 0. Test is automated by using a suitable accessibility evaluation tool such as EvalAccess2. 0. This tool provides different options to evaluate the accessibility such as we can evaluate a single page or whole site or HTML mark-up.This tool is freely available online and generates complete error report after evaluation. The tool matches the web contents of MapQuest with WCAG1. 0 standards and then outcome the results. These results are further discussed and analyzed in next chapter. Goal Question-Metric (GQM) approach According to Teles de Lima (2007), “GQM approach is used to define what was going to be evaluated and the main objective of this approach is that measurement should be goal-oriented”. In this approach explicitly a goal is defined then this goal is refined in the form of several questions that divide the goal into its major components.Then each question is measured on the defined scale. (Teles de Lima, 2007) We also used GQM approach to evaluate the accessibility of MapQuest. In this evaluation test we have defined a variety of tasks given in next section and each task is performed by elder persons individually using the application MapQuest. After completing the task, a questionnaire related to the accessibility issues found in MapQuest is given to the same individual and requested to answer these questions. Thus answers of the given questions are used to evaluate the accessibility of MapQuest and to declare the results.In our work the GQM goal is Table 4. 1: GQM Goal in this Test Fig 4. 3: Flow Diagram: Evaluation process for Accessibility of Web based GIS (Teles de Lima, 2007) Plan to evaluate the 2nd research question Data regarding second question is gathered through interviews. Interviews are taken from the developers working in any organization on web based GIS applications. The interview is supported with different types of questions. After asking questions we judged what are the expert observations, comments or experience about that. What they suggests about that thing. It is our hypothesis that ?User Centred design (UCD) can be helpful for UI designer to enhance the accessibility of web based GIS”. But data regarding this approach is collected through interviews and based on the analysis of that data we can say that our hypothesis is true or false. Interview questions and their responses during the interviews are given in appendix E. Selection of participants As observations and comments of participants have a major role in the evaluation of a test so some caring attitude is required in the selection of the participants. In our test plan we have selected two broad categories of users.To evaluate the accessibility of MapQuest we have selected elder people who have used this application or at least have knowledge about GIS applications. Before conducting the test particulars of each participant are noted which include the name, age, knowledge about GIS and any physical or mental disability? The second category of participants consists of the designer who is working and developing web based GIS applications. In this case particulars of each individual are also noted which include name, organization and experience in this field. Table 4. 2: User’s category Environment and Equipment used in the testThe test is conducted in quiet and user friendly environment and this environment is supportive in evaluating the better results from the users. Both hardware and software are utilized in this experiment. Hardware includes the personal computer, mouse, keyboard, printer, camera and software includes web browser and evaluating tool(s). Other accessories include paper, ball point etc. Camera is used to take the snapshot of each participant at the end of the task to create the originality in the test. Tasks and Task Explanation Tasks that are performed by users in Accessibility Test are given below. Table 4. List of Task 4. 8 Google Maps ; MapQuest usability testing Usability testing plays a vital role in this dissertation, because it helped to observe different users while interacting with Google Maps and MapQuest. To get an in-depth knowledge of usability issues of Google Maps and MapQuest, usability tests were conducted on some volunteers. The tests were conducted in ‘Newcastle Academy of Business Technology (NABT)’, Newcastle upon Tyne. Many students were interested for participation but only five students were selected in which two of them were expert user of GIS applications and three of them were novice users.First the usability tests were conducted on the participants. The participants were given 3 tasks to complete on Google Map and MapQuest applications respectively. The test was conducted in computer lab and it was free from any kind of disturbance. All the systems were thoroughly checked for internet connectivity and other problems that may occur. To avoid bandwidth difference same computer was used by every participant one by one. The NABT computer systems had CamStudio (Free Screen Recording Software) installed in it for recording the time each participant took to complete the task.Before the commencement of usability tests the participants were carefully guided about the test they were going to attend. The participants were well aware of the GIS applications that they were going to use and about the task they have to perform. Think aloud technique was used for testing in this study and users have been informed to think aloud while performing the tasks. During the test, authors observed each user while performing the tasks. Think aloud method (Nielsen 1993) was useful in this testing, and much information was collected by the authors regarding interactivity of users with web based GIS applications.By using this technique, the authors both observed the users and could share how the users perceived the web-based GIS applications. Before starting the test, the tasks list was distributed to the user for performing the tasks. After finishing every task the time taken to finish it was written down by the author. The usability test task list included three task and each participants were asked to complete the task in both the GIS applications, that is, Google Maps and MapQuest respectively. The task list is listed below: * Searching an address Getting direction from one address to another * Zooming and panning of map 4. 9 Questionnaire After successfully completing the usability test, those five participants were handed the questionnaires. The questionnaire was taken from the research study of Khan and Adnan (2010). The questions in the questionnaire are related to the criteria that were selected by the authors such as effectiveness, usefulness, user reaction, consistency, architectural and visual clarity, and functionality. Three of these criteria are presented by Koua et al. 2006), which suffice for assessing user performance and satisfaction with the GIS applications while the remaining three criteria are adopted from IS;T guidelines, which covers its web-based nature. All five participants returned the questionnaire duly filled. CHAPTER 5 RESULTS / ANALYSIS This chapter introduces the results achieved through the evaluation methods. The first phase includes method that we have used to evaluate the accessibility of MapQuest is the content based evaluation method and the second one is called evaluation based on user interaction.Interview results which show that the UCD approach is supportive to enhance the web accessibility are also included in this chapter. The second phase includes all the results from the usability tests, questionnaires and interviews. 5. 1 First Evaluation- Standard Review Method: In this experiment work, Accessibility evaluation of the site MapQuest has been done by using the tool EvalAccess 2. 0 which uses the standard web content accessibility guideline 1. 0 as a foundation. But this is a content based evaluation method and this is not sufficient to evaluate the accessibility of MapQuest.So we have used another method that is user interaction based evaluation that is discussed in the next section. Both evaluation methods can provide useful results. This tool demands three types of input that is ‘Insert the URL you want to evaluate’, ‘Depth’ and ‘maximum number of URLs you want to evaluate’. After giving the URL of GIS site MapQuest, depth value 3 and maximum number 15, the tool took some time to evaluate the desired site. In depth 1 we selected 1 page, depth 2 we chose 3 pages and finally from depth 3 we also chose 1 page. Thus a total of 5 pages of MapQuest have been taken under consideration.The results thus achieved have been further summarized into tables. At the end the WAB score of 5 pages has been calculated mathematically. The following tables (table 5. 1, 5. 2, 5. 3) show occurrence of errors against each checkpoint. For example checkpoint 1. 1 indicates 24 times the violation of this rule at different lines of the page under depth 1. Depth 1: Table 5. 1: Accessibility evaluation under depth 1 (EvalAccess, 2009) Depth 2: Table 5. 2: Accessibility evaluation under depth 2 (EvalAccess, 2009) Depth 3: Table 5. 3: Accessibility evaluation under depth 3 (EvalAccess, 2009)We can organize all the errors that have been diagnosed in above tables into priority levels with reference to the page under different depths. So the following tables show the errors in three defined priority levels. Depth 1: Table 5. 4: Errors with respect to priority levels under depth 1 Depth 2: Table 5. 5: Errors with respect to priority levels under depth 2 Depth 3: Table 5. 6: Errors with respect to priority levels under depth 3 The above results are further organized as how many checkpoints have been violated against each priority level under different depths.The following results have been calculated from tables (5. 1, 5. 2, and 5. 3). As we have already mentioned that priority 1 contains of total of 29 checkpoints, priority 2 contains of total of 40 checkpoints and priority 3 contains of total of 22 checkpoints. For example page 1 under depth 1 has violated one checkpoint out of 29 in priority 1, three out of 40 in priority 2 and two out of 22 in priority 3. Table 5. 7: Violations against priorities From table 5. 7, we can easily conclude the accessibility according to this way of measuring accessibility.For example page 1 under priority 1 has violated 1 checkpoint out of 29 so in percentage it will be calculated as (1* 29 /100). The percentage value indicates the number of barriers on each page. This value is inversely proportional to the accessibility of the page. This means that decrease in this ratio increases the accessibility and vice versa. Statistics in table 5. 8 show that the percentage of accessibility barriers lies between 3. 448 (minimum) and 9. 091 (maximum). Minimum barriers exist on priority 1 and maximum barriers exist on priority 3.As the site does not satisfy all the checkpoints at priority level 1, it does not meet conformance level ‘A’. There are barriers at priority level 2 and 3, thus it does not meet conformance level ‘AA’ and ‘AAA’ either. But one checkpoint has been violated in priority level 1 so approximately it meets the 97% of the conformance level ‘A’. Similarly it meets 92% and 91% conformance level of ‘AA’ and ‘AAA’. Accessibility barriers on web pages in percentage are shown below. Table 5. 8: Accessibility Barriers in percentage with respect to priority levels The following graph shows the barrier and accessibility of MapQuest in percentage.For example in priority 1 the barrier ratio is about 3% and accessibility at that level is 97%. Graph 5. 1: Comparison of barrier/accessibility of MapQuest The following graph shows the conformance level of MapQuest in percentage. Graph 5. 2: Conformance level of MapQuest Table 5. 9 shows minimum barrier ratio at depth 3 and maximum barrier ratio at depth 1. By removing the barriers at depth 1 we can enhance the accessibility of MapQuest. The accessibility of web pages on each depth is given below. Table 5. 9: Accessibility Barriers in percentage with respect to depths 5. 2 WAB Score of MapQuest:In the calculation of WAB score we have taken a total of 5 pages of MapQuest site and the distribution of those pages is such that 1 page is taken from depth 1, 3 pages are taken from depth 2 and finally 1 page is taken from depth 3. Weightage (Wv) of priority 1 is 3 times more than priority 3 and Weightage (Wv) of priority 2 is 2 times more than priority 3. From the above table 4. 7, we have calculated the WAB score of MapQuest as follows. Here, total no of pages checked (Np): 5 Page on depth 1: (1/29 * 3 + 3/40 * 2 + 2/22 * 1) (0. 103 + 0. 15 + . 091) (0. 344) Pages on depth 2: Page on depth 3: (1/29 * 3 + 2/40 * 2 + 1/22 * 1) (0. 103 + 0. + . 046) (0. 249) According to WAB formula as given in figure 3. 1, WAB Score of MapQuest = (0. 344) + (. 887) + (0. 249) / 5 = 1. 48/5 = 0. 296 Note: WAB score indicates the number of barriers out of 5 pages of MapQuest site. Zero WAB score indicates maximum accessibility of the site and six WAB score in this case indicates minimum accessibility of the site. MapQuest lies at ‘. 3’ between scale 0 and 6. 5. 3 Second Evaluation – Goal Question Metric Method (GQM) With this method, evaluation of accessibility of MapQuest was done with respect to elder people, what they perceived as difficult in accessing and interacting with this application.Each user was requested to perform a task that has been specified (see chapter 4 under table 4. 3) so that they could get some understanding of the MapQuest application. After this a Questionnaire was filled by that user. The questionnaire is designed in such a way that it has been partitioned into four categories. Each category contains a different range of questions with maximum 6 and minimum 1. These questions cover different aspects of the accessibility of MapQuest. Scale from I to 5 has been specified and the user has to answer each question by using that scale.If users do not want to answer any question then they can use the option not applicable (N/A). After performing the tasks and answered the questions individually, we analyzed the questionnaire response to get quantitative results. That result is then populated differently to get clear idea of the statistical data gathered. We have used tables, graphs and diagrams to elaborate Questionnaire responses and to evaluate the accessibility of MapQuest with respect to the user experience of interaction with the application. 5. 4 Over All response of Questionnaire Different users mark different rating against each question.Quantitative data thus received is passed through mathematical formula to get useful information. Table 5. 10: User‘s overall response in the Questionnaire The following graph gives another view of the user‘s overall response to the Questionnaire. Graph 5. 3: User‘s overall response to the Questionnaire The questionnaire has been divided into four categories and the following graph shows the user‘s response concerning each category. Graph 5. 4: User‘s response against each category The above statistics in table 5. 10 are further organized into percentage against each category.The following table gives more precise and understandable results with respect to each category. Table 5. 11: User‘s response in percentage against each category Above facts and figures can be further summarized into two poles that are ‘Disagree’ and ‘Agree’. The middle point of this pole is ‘Average’. If we assume ‘Strongly Disagree’ and ‘Disagree’ value to one that is ‘Disagree’ and same ‘Strongly Agree’ and ‘Agree’ value to one that is ‘Agree’ and leave the ‘Average’ value unchangeable. The following table presents the response of the users on this limited scale. Table 5. 2: User‘s response in dichotomous form From the above table it is clear that 29% of users disagree with the opinion that the site is not perceivable and 50% agree. Similarly, 22% users disagree and 59% agree under operable category. In understandable category, 25% users disagree and 58% agree. 17% users disagree and 83% agree under compatible category. This means that more than 50% users agree that MapQuest displays the features of the category operable, perceivable, understandable and compatible as percentage of agree is more than disagree. During this analysis we have kept the average value untouchable.But if we include average value to the agree pole then results against each category will also be different. We assume percentage value of ‘Disagree’ in table 5. 12 as barriers for the users in accessing MapQuest. So we can plot a graph between barriers and accessibility of MapQuest against each category. Graph 5. 5: Accessibility/Barrier ratio against each category Note: Tasks (see table 4. 3) were performed by six elder people and all the results related to the barriers and web accessibility as shown above were calculated based on the observations and questionnaire filled by those people. . 5 Role of UCD (User Centred Design) in Accessibility To probe the second research question of our research study i. e. what is the role of user Centred design in accessibility we used the method of interviewing people who are involved in development of GIS applications. For this purpose we visited one of the leading GIS solution providers of United Kingdom, ESRI (UK) Ltd, Aylesbury, which have branches in 4 different countries and clients from 30 different countries. The company provides both desktop and web solutions to its clients.To prove our hypothesis that user Centred design is helpful in enhancing accessibility, we wanted to investigate whether they use any User Centred approach, what are accessibility problems of their product and what approach they are using to handle accessibility problems in the products developed by the company. We visited ESRI (UK) Ltd, Aylesbury office on 3rd September 2010 and met Development Manager Mr. Tony Wilsson. First he gave a brief presentation about their development process and the role of every member of the team in the development process.After that we asked some questions about the role of User Centred Design in the development process and handling of accessibility issues. Interview results are given below and sample interview is given in appendix E. 5. 5. 1 Interview Results 1. Responding to the question whether any User Centred approach is used in the company‘s development process; the Development Manager said that they are using two approaches of User Centred Design to involve users in development process. The company use Participatory Design approach by forming a REFERENCE GROUP as part of the team which consists of different types of users.They also use Contextual Design UCD approach by including in their development team a BUSINESS AREA SPECIALIST (BAS) who gathers product related data from users and other related people in the field. 2. Answering question if there is any other approach that is more suitable than UCD in development process; the Development Manager said that the company has tried other approaches but has found involving users in early stages of development more suitable and fruitful than any other approach because it is very difficult to incorporate user‘s requirement at later stages of development. 3.When asked about at what stage users should be involved in the development process, the Development Manager said that it is useful to involve users at an early stage of life cycle development of the product. 4. When asked about what are the accessibility issues of the product developed by the company, the Development Manager replied that they face two kinds of accessibility issues, firstly the common accessibility problems that are faced by people with disabilities to use computers and web and secondly the lack of knowledge among many of the users about Geographical terminologies. . When asked how they handle these Accessibility problems faced by users, the Development Manager replied that they use the UCD approach during development of product to gather accessibility problems of the users and Support approach after the product release to gather the accessibility problems of the users. After gathering these accessibility problems, development team incorporates solutions in the product that is being developed. . When asked about Support approach, the Development Manager replied that users can point out errors in the product and communicate wishes and requirements of some extra functionality in the product by contacting the support manager. The support manager then ranks these errors and wishes on a scale of 1 to 3 by observing the fatality of error and need of functionality. 7.When asked which approach is better to handle problems regarding accessibility, Development Manager said that UCD is better approach to handle the accessibility issues due to report of that problem at early stage and also it is easy for developer to include handling of that issue at early stages of development. 8. When asked that what measures are planned to be taken to handle accessibility issues of the product, the Development Manager replied that the company has planned to widen the base of Reference Group and include persons having different types of disability to handle most of the accessibility issues.They have also planned to educate the users about geographical terminologies. 5. 6 Section 2 This section of dissertation depicts the usability test results. It also includes the comparison of the aforementioned two GIS applications, that is, Google Maps and MapQuest. Results from the questionnaires are also presented in this part. 5. 6. 1 Results achieved from usability tests: For the usability test, the author selected five students as participants from Newcastle Academy of Business Technology, Newcastle upon Tyne. The tudent participants were further categorised into two that is, beginners and experts, on the basis of their knowledge about using GIS. Two of the participants were expert users and three were beginners and had limited knowledge in GIS applications. The participants were given different task and the time required to finish the task were recorded down. Below mentioned table 5. 13 shows the task list which the participants were asked to complete on Google Maps and MapQuest. Table 5. 13 Task list Task 1 was to search for a particular address on the maps for example, ‘Wingrove Avenue’.Task 2 was to find the distance and direction from Wingrove Avenue to the Newcastle Academy of Business Technology. And final task was to zoom and pan the map at specified locations. Table 5. 14 depicts the results that were recorded for the test conducted. The table lists the time taken to complete each task in both the GIS Applications. The participants were categorised into two that is beginner and expert. Table 5. 14 Results of usability tests Graph 5. 6 Average Time taken to finish each task The above graph shows the average time taken to finish the task allotted to the participants.The graph clearly states that both the GIS application works in similar way and there is very little difference in the performance they render to the user. It was also observed that certain tasks were easier to carry out in one GIS application and certain other task were more quicker in the other. This study helped to understand that both GIS applications have some usability issues and if we remove those issue then the application will be able to perform much better and users will feel more comfortable with it. 5. 6. 2 List of observation from usability tests After conducting the test and comparing the results certain points were observed.They are listed below in tabular manner in table 5. 15. The table contains the task and observations from the results. Table 5. 15 List of Observation from test conducted 5. 6. 3 Results from the questionnaires The questionnaire prepared had 24 questions and all of them were close-ended and therefore was easier for the participants to select one option for each question. For ease of use, the questionnaire was designed according to Likert scale that is having 5 options for each question. It was kept in mind that the questionnaire should be easily understandable and not time consuming and frustrating.The same five participants from ‘Newcastle Academy of Business Technology’ were used for this survey as well after the completion of usability test. The questionnaire was designed to find out the level of satisfaction users get after using both the applications. The soft copy of questionnaire is included in Appendix F. The scale of questionnaire was based on Strongly Agree, Agree, Not Sure, Disagree and Strongly Disagree. The questionnaires were duly collected from the participants after completion and the results were analyzed. The results are depicted in below Graph 5. 7. Graph 5. 7 Response of questionnaireThe analysis of questionnaire response concluded that the users preferred Google Maps over MapQuest. The questionnaire responses included 41 and 24 ‘Strongly Agree’ for Google Maps and MapQuest respectively. 51 users responded ‘Agree’ for Google Maps and 42 for MapQuest. 12 users responded ‘Not Sure’ for Google Maps and 11 users for MapQuest. Similarly 13 users checked ‘Disagree’ for Google Maps and 27 for MapQuest and 3 users responded ‘Strongly Disagree’ for Google Maps and 16 for MapQuest. It is clear from the above graph that participants responded more ‘Strongly Agree’ and ‘Agree’ responses for the Google Maps than MapQuest.Also participants responded more ‘Strongly Disagree’ and ‘Disagree’ responses for the MapQuest than Google Maps. When we talk in terms of usability, strongly agree are symbolizing positivity and strongly disagree symbolize negativity. 5. 6. 4 Comparison of Google ; MapQuest The response of questionnaire were recorded and analyzed. The below table 5. 16 depicts the response for the 24 questions present in the questionnaire. The questionnaire was inspired by the research work done by Khan and Adnan (2010). The questionnaire was divided into certain categories to better evaluate the GIS application.They are listed below: * Question 1-7 Effectiveness * Question 8-13 Usefulness * Question 14-18 User Reaction * Question 19 Consistency * Question 20-22 Architectural and Visual Clarity * Question 23-24 Functionality Table 5. 16 Questionnaire response Graph 5. 8 Comparisons of Google ; MapQuest The above graph 5. 8 compares both GIS applications on the basis of categories aforementioned. Out of 40 questions asked to 5 respondents on effectiveness of GIS application, 15 strongly believes Google Map is much better and on the other hand only 6 strongly agree that MapQuest is better.Similarly we can figure out that Google Maps scores higher in almost every aspect that the both GIS applications were evaluated. It is very clear from the above graph that Google Maps are preferred more. CHAPTER 6 DISCUSSION/ANALYSIS AND VALIDITY STATEMENTS This chapter contains discussion and validating statements. In the first section ‘Discussion’, the methods used are discussed to evaluate the accessibility of MapQuest; highlight various accessibility issues observed during user interaction. ‘Validating statements’ gives a brief discussion on the methods used to evaluate the accessibility and UCD approach.The second section contains a discussion about the interface of both applications regarding adopted criteria for usability evaluation. SECTION I 6. 1 Discussion The main aim of this report is to evaluate and highlight accessibility issues of web based GIS applications with respect to disabled persons and then propose UCD approach to enhance the accessibility of web based applications. To narrow down the research space and to get a deeper understanding about accessibility issues we have selected a broadly used web based application MapQuest as a case study. We have used a systematic approach to evaluate the accessibility of MapQuest.We have used two techniques/methods, ‘Standard reviews method’ and ‘Goal questionnaire metric method (GQM)’ to evaluate the accessibility. The first one is a content based evaluation method and second one is an evaluation based on user interaction. After discussion with the supervisor it has been realized that a single evaluation method is not sufficient. Both evaluation methods can give useful results that complement each other and enhance understanding of the issues involved and thus we can find out accessibility issues with different perspectives. In the first evaluation method we have used an automated tool called EvalAccess 2. that gives the results based on the web content accessibility guideline (WCAG). Many similar tools are available on the net such as WAVE, Hera etc, but we have selected this one as it is freely available on the net and further it meets our requirements. Through this tool we have calculated web accessibility barriers at different priority levels (1, 2 ; 3), conformance levels (‘A’, ‘AA’, ‘AAA’) of MapQuest and finally calculated the WAB score of MapQuest through proposed WAB formula. Based on the WAB score we can rate MapQuest and also different web sites on the net.Although standards regarding web accessibility have been developed with respect to the user‘s needs, the results produced by this tool are not sufficient and satisfactory so we have used a second technique to get maximum outcome from the users. We have observed the barriers in web accessibility in real time when the users were interacting with the application. The hurdles in accessing the web site are further discussed later specifically with respect to disabled persons. We have used a dual approach to evaluate the accessibility with different perspectives and the results achieved from each method can be compared to validate the findings.It is not necessary that both evaluation methods produce the same results. Each method has its own benefits and drawbacks. There are four principles of accessibility and we will discuss each one individually and try to mark the accessibility issues facing users in accessing MapQuest under each principle. 6. 1. 1 Perceivable Information in web based applications should be perceivable, adaptable and distinguishable especially in those applications with users interact daily or fairly often. Perceivable means that information and user interface components on the web page should be presented in such a way that users can erceive them, not only normal users but disabled people also. If we observe the results in table 6. 12 achieved after user interaction with MapQuest then we can conclude that 50% users agree that the site is perceivable and 29% disagreed with this opinion. But 21% users have responded in average against this principle. ‘Text alternative to a non-text object’ should be provided so that it can be changed into other forms according to the user needs. MapQuest has generally followed this rule but there are some places where this rule has not been followed.For example, at the top of the site on the left corner there is an input field ‘Search the web’ which has label missing. During interaction it has been observed that when the user opens this site (http://www. mapquest. com/) and wants to see the map of India he/she cannot succeed unless they selects the ‘All counties’ option from the international list. Many users during interaction have raised this issue and feel it is a difficulty. So by default this option should be ‘All countries’ instead of ‘USA ; Canada’ so that users belonging any region can access their maps easily.The location of this option should be prominent and name is also creating confusion for the users in understanding this feature. During interaction it has been observed that whenever a user clicks on the map and holds down the ‘left mouse button’ to move the map up and down, the shape of the cursor does not change in browser ‘Google Chrome’. Although it should be changed from to because anchor shape gives some meaning to the user and it is the de facto standard. Users are satisfied that contents on the site are distinguishable.Foreground and background colour scheme have been chosen properly to make the contents and user controls prominent. 6. 1. 2 Operable It is necessary that user interface components and navigation should be operable. Operable means that user components can be accessed through different ways. During interaction it has been observed that aged people that have not full grip on the mouse are feeling difficulty in accessing this site. 59% users said that site is operable and 22% have disagreed with it. We observed the following issues during user interaction with MapQuest.Through arrow keys user cannot move up and down, left and right within the map area. Users need mouse to do this which is a hurdle especially for elder people. The second thing that has been observed was user has to press many tabs to go to the ‘End country’ whenever they want to get the directions through MapQuest. Users can not ‘Zoom IN’ and ‘Zoom OUT’ the map through mouse scroll button. They have to use ‘Zoom control’ for this purpose so this thing is also a barrier in accessing this site. Google map has eliminated all these deficiencies to make the site maximum accessible for the users. . 1. 3 Understandable Most of the information and user interface controls on MapQuest are understandable for the users but still the following issues have been raised by the users during interaction and they are also creating ambiguity for the users. 58% users have the opinion that the site is understandable and 25% have disagreed with it. When the user wants to see the map or direction then information ‘Find a business’ is presented under the heading ‘Get map’. Here user takes some time and feels difficulty in understanding this label.Further, icons (Find airport, find category) have been given that are not understandable and give no meaning to the users within this task. This information is irrelevant and confusing for the users. 6. 1. 4 Robust According to this principle, web content can be retrievable by using all types of user agents and assistive technologies (Henry, 2005). During interaction it is observed that MapQuest gives proper feedback to the users and assists the users during a task. This application is also opened within different browsers such as ‘Google Chrome’, ‘Mozilla Firefox’ and ‘Internet Explorer’ to check its compatibility. 3% users are satisfied with this feature and 17% disagree with it. When this application has been opened in different browsers the following issue is observed. 6. 2 User Centred Design and Accessibility From Interview response it is evident that the developer who was interviewed agreed with the role of User Centred Design in enhancing the Accessibility. After probing the first researcher question we encountered the fact that normally used tools for web content evaluation cannot identify some problems aced by users having some disability. Even some Guidelines given by WAI (Web Accessibility initiative) are directed to be checked manually, thus limiting the role of automated evaluation tools for web accessibility. Developer that was interviewed therefore agrees that user participation in design cycle is very helpful to enhance the Accessibility of the content being developed. 6. 2. 1 User Centred Design During the interview, the Development Manager Tony Wilsson gave details about how company uses UCD in development process.He told that Participatory Design and Contextual Design are used to involve the user in development of the GIS application by Reference Group and Business Area Specialist. The company also gathers user responses after the release of the product by Support section getting in touch with users and vice versa to record anomalies and suggestions. So the Development Manager was in position to tell the difference between the results gathered form UCD approach and other approaches. The Development Manager ranked UCD as more suitable and result oriented approach to investigate user‘s problems regarding the product.The company uses two methods to apply UCD approach * Company has included Reference Group as part of its development team to include users as part of its development process. Participants are consulted frequently to discuss the problems that may hinder the accessibility and usability of the product that is going to be developed. * Company has a specialist called Business Area Specialist (BAS) who has also previously used the GIS applications to interact and collect user‘s preferences and expectations regarding the product and then convey these preferences and expectations of the users to developers. Company uses support function that includes one farmer developer and feedback from customer. This support is regularly included in development projects at the start up discussions. 6. 2. 2 User Centred Design and Accessibility Problems During Interview Tony Wilsson the Development Manager told us how User centred Design is helpful in identifying the user‘s issues and handling that issues in time to make more accessible and usable product. According to Tony Wilsson, company uses both UCD and conventional methods to identify and then esolve the accessibility issues and find UCD approach is better in identifying the accessibility issues. The benefit of using UCD approach is that the user‘s problems regarding accessibility are identified at early stage and are easier to handle at that stage. The main problem faced, when accessibility issues are identified at later stages or after release of the application either through conventional user response or through accessibility evaluation tools, is that at this stage it is very hard to incorporate the solutions of that accessibility issues.Tony Wilsson agrees with above problem because according to him the company has used both methods as identification of accessibility issues. Also Tony Wilsson confirmed that some problems that cannot be identified by automate accessibility evaluation tools are identifiable by using UCD approach. So User Centred Design approach helps in enhancing accessibility of the GIS applications in the following way. * UCD approach helps identifying accessibility problems that cannot be identifies by automated accessibility evaluation tools and by addressing these problems final application should have increased accessibility level. UCD approach identify accessibility issues at early stages of development process so development team has more time to address that issue and also at early stage it is comparatively easy to incorporate the solution of that accessibility problem. 6. 3 Validity Statement Author has used both qualitative and quantitative approaches in this research thesis report and the validity of these methods used is checked by criteria proposed by Guba and Lincoln (Trochim, 2006). The criteria to be followed include Credibility, Transferability, Dependability and Conformability. . 3. 1 Credibility Author used symmetric approach to gather the results for research project. Firstly an evaluation tool is used to gather data about accessibility of web based application through content based evaluation. Since this evaluation method is not comprehensive and requires some manual testing so we conducted Questionnaire for people with some disabilities. So both methods are complementary for each other. Participants of the questionnaire are people with disabilities and can identify true problems faced by users with isabilities during usage of web based GIS. 6. 3. 2 Transferability Author selected very commonly used web based GIS application MapQuest for our case study and objective of using this application is that results could be generalized to other web based GIS applications. The methods used in this research project can be used for any web based GIS application and in fact for any web content. The results of the Accessibility evaluation may vary according to measures taken in that application to overcome accessibility barriers and following WAI guidelines.However one can use different methods to check the role of User Centred Design to enhance the accessibility of web based application. 6. 3. 3 Dependability Dependability is the effect on the result due to changing in timing and setting (Trochim, 2006). Author conducted content based evaluation with the same setting of the web based GIS application selected for case study. Then Author conducted the questionnaire session of evaluation with the participants who can speak English well so they can understand and respond to the questionnaire easily and fluently.There was not too much difference in the timing of the conduction of interview for each participant so the setting and the time is not much changed. 6. 3. 4 Confirmability This criterion ensures that other researcher can confirm the results gathered in present report. Confirmability is the degree to which the results gathered in underlying research are checkable by other researchers. Author formatted the report in comprehensible section that could enhance the confirmability.Results and findings gathered from different methods are placed in separate chapter that make it easy for researcher to confirm the results. The design of the experiment and questionnaire are placed in different section and sample questionnaire and interview responses are added in the appendixes for further consultation. 6. 4 Section II An analysis, conducted over the usability test and questionnaire helped the authors to find various usability issues in the web-based GIS applications through analysing its results, which discusses the problems regarding bothGIS applications in the context of adopted criteria for this study. 6. 4. 1 Effectiveness According to Koua et al. (2006) effectiveness deals with application functionality and observations of user performances and experiences of the tasks. It also deals with the gathering of data and any parameters available to complete a task. The time taken by each task, errors rate, satisfactory performance of the application, the correctness of outcome of the tasks, meeting the user requirements and its response against mistakes determined the effectiveness in the questionnaire.In the case of Google Maps, it was appreciated by the users as it was effective and requirements could be fulfilled successfully by using it. The use of the simple search boxes helps in saving time during the search of a specific location but it gives too many options in search results, which makes it challenging to select the right option. Error messages are displayed but are not always in informative form. Although it is difficult for users to locate in Google Maps, overall the performance of the Google Maps was proved valuable by the users.Whereas the MapQuest have separate text boxes for “direction”, “locations” and “businesses” which are required to be filled it in correctly, else no results are displayed and user has to switch to a new search. It is thought to be effective but not entirely as it is time consuming. Although help topics are provided for guidance of users but do not offer any help to view streets and directions. MapQuest proved to be helpful as requirements can be fulfilled by using this application. 6. 4. 2 Usefulness According to Koua et al. (2006) usefulness is all about user’s expectations and demands in successful performance of different tasks.It looks as though either application is supportive for users’ goals and tasks, users can easily understand and interpret the application results, whether it’s flexible according to user expectations, simple to use and the number of steps required while carrying out a task. On comparison of Google Maps with the MapQuest, positive feedback on different tasks was obtained that proved the former to be more useful as locating a specific place using less information is very effective. Using the option of “Get Directions” for Task 1 proved very informative after obtaining the results as it gave step by tep directional details. The problem involved is that the links provided are not very visible and attractive. Although users found the links to be very much visible, in the case of MapQuest but overall the interface was seen to be very messy, hence users preferred Google Maps of MapQuest. Too many boxes are to be filled by users and sometimes they miss out on selecting the region which leads to frustration. But those who do not have any idea about searching find it useful as it provides an interface in which they can easily write address, city, state and ZIP code etc.In MapQuest has an edge over Google Maps in regard to the print option. 6. 4. 3 User Reaction According to Koua et al. (2006) user reaction concerns their opinions, views, attitude and preferences towards an application. The questionnaire calculated the level of easiness for new users, its use over long run and its recommendation to a friend. The reaction of the Google Map users was positive and it was learnt that they began to feel familiar with it and learn it quickly. Most users recommended using Google Maps as reaction towards MapQuest was not up to the mark.Since the new users found it difficult to carry out a task in the first go as well as did not appreciate advertisements and other sponsored links on the website. 6. 4. 4 Consistency According to Usability Guidelines (2009), consistency means navigation through different pages of the website should reflect the home page. Icons, words and phrases should be consistently described for the same item. In a consistent system the same basic steps are used for different tasks which make it easier to learn. According to the authors Google Maps and MapQuest, both are equally consistent and need to improve their consistency.In the case of MapQuest while navigating through different pages, inconsistency crops in by the use of advertisements and sponsored links as these appear differently on every page. 6. 4. 5 Architectural and Visual Clarity According to Usability Guidelines (2009) a well organised site from a user’s perspective, i. e. the usage of colours, optimum white space on the page, and exclusive of unnecessary animations means it has architectural and visually clarity. In the author’s opinion, the main page should be impressive as it is the first interaction point of any application with users. For that