Evaluation of the HUman factors and ergonomics Society (HFES) Website
Overview
I completed an Ergonomics Assessment course in which the primary course purpose was to learn about and apply various Human Factors and UX methods to small and large projects. One course requirement was a group project that required each group to evaluate an existing product/system with methods we learned in class. My group chose the Human Factors and Ergonomics Society (HFES) website. We conducted a survey to collect data on the areas that may need to be redesigned. Based on survey results, we focused our efforts on website navigation, including the navigation menu, and the job search function. The following methods were used: user needs analysis, open card sort, hierarchical task analysis, quantitative situational awareness, scenario-based design, task analysis, and heuristic evaluation.
My Contributions
I performed a cognitive analysis on the navigation menu using an open card sort. I also performed an independent heuristic evaluation on the job searching functionality.
Team Members
Michaela Case, Jesseca Taylor, Mary Luong, Brannan McDougald, Dan Schantz
Tools and Methods
online open card sort, heuristic evaluation
Card Sorting Process
I conducted an online card sort using menu names from the HFES’ navigation menu.
The card sort was created and conducted on websort.net.
Menu links were obtained from a generated site map of the website’s navigation menu and used.
Menu link names and descriptions of the corresponding webpage content were entered into an item list in the card sort.
Using various sources of data, the following results emerged:
I determined which items participants tended to group together and merged the categories that were assigned to those items.
Example of categories merge: “Careers,” “Careers in HFES.” “Career Center,” “Job Seekers,” “For Job Seekers,” and “Job Bank” were merged and renamed “Jobs”
I identified items within a category that had the highest agreement among participants. I kept those items grouped with that category and remove the other items.
Outcome
After analyzing the data, I reduced the top-level navigation menu items from 22 to 9. The images above are some examples of the categories and and their sub-menu items. Category names are in bold font. The items highlighted in yellow were the actual top-level menus displayed on the homepage when the card sort was conducted.
The card sort study results suggested that fewer top-level menus better fit users’ expectations as to how the navigation menu should be organized. This study’s results were consistent with results from a user test conducted by my team. In the user test, participants gave lower workload scores, were more efficient at finding information, and preferred the prototype with the new menu structure compared to the prototype with the existing menu. Furthemore, HFES has changed its navigation menu to have fewer top-level items. Overall, these findings suggest my card sort study produced valid results.
Heuristic Evaluation Process
Another evaluator and I independently conducted a heuristic evaluation of the HFES website (hfes.org). We evaluated several functions related to job searching on the website. The functions evaluated were logging into the career center, searching for jobs based on zip code, searching for jobs based on state(s), building a searchable resume, and saving a job posting. We evaluated the functions mentioned above against a set of heuristics developed by Jakob Nielsen.
List of Heuristics Used in Evaluation
The evaluation revealed that hfes.org had several successful job searching features.
Consistency and Standards
Users could restrict their job search to specific states. Search results appeared to be consistent with other location-based search features on other websites: Searches returned only job postings from specified states. The figure below shows an example of website’s display after searching for jobs in Maryland and Kansas. In addition to searching by state, searching by area of interest worked as expected.
Visibility of System Status
After a job posting is selected to be saved, the status “save job” changes to “saved.”
Results from job search by location
Although the evaluation revealed success areas, the evaluation did uncover some potential issues. Details on a subset of the issues are listed below.
Aesthetic and Minimalist Design
Approximately 50% of right side of the webpage is unused whitespace. On several pages, users must scroll to read all of the information.
Recommendation: Use the right side of web page, which may eliminate the need to scroll on some pages.
Visibility of System Status
The website did not provide a status update or message for users who failed to enter login information correctly. Therefore, users could navigate and perform job search functions, such as save job postings, without being logged in. Guest users who believe they were logged into the career center needed to take further steps to save the job posting list to prevent losing their progress. Guest users did not see the warning to log in and save jobs before closing browser unless they navigated to another page.
Recommendation: For guest users, disable the option “save a job” or when they click “save a job” tell them that they are not logged in and must do so to save jobs.
System status of Employer Login is not very salient, and the status is not in close proximity to where login information is entered.
Recommendation: Highlight the login status by using a larger font size and move the status to the left side of the screen to increase its visibility.
Help Users Recognize, Diagnose, and Recover From Errors
No status is provided indicating when users fail to enter login information correctly.
Recommendation: Provide a feedback message that indicates login information was submitted. Display a message stating that login was unsuccessful if users enter incorrect information.
Consistency and Standards
Search by zip code returns postings from places far away from entered zip code. For instance, searching for jobs in the 27606 area (Raleigh, NC) returned jobs 150 miles away from Raleigh.
Recommendation: Modify search by zip code to show only jobs located in the specified zip code and radius.
User Control and Freedom
Users are provided with the option to search by zip code, but cannot choose what distance in which to restrict the search.
Recommendation: Restrict users from searching by zip code or provide a function for users to indicate a distance (i.e., radius) to restrict the search.
Outcome
The results from this evaluation were consistent with data collected from other evaluations. In a questionnaire, users reported difficulties using the job searching function on the website, and the evaluators experienced problems while navigating and using the job searching function as well.