ANSWER 1: Confidentiality is different from the user’s right to privacy, it refers to how data about the participants are stored.
Field observations: Conducting usability tests in the place of the user or in uncontrolled setting to understand the real usability of the system. Here the modes of collecting data are writing down the observations and recording audio and video with the user’s consent. In this case the ethical rules to be followed are to pre formulate alternate names for specific names such as using P1,P2,P3 instead of writing down the name of the patient. Also recorded patient data will be coded based on the separate name based scheme so that other people cannot understand what the data means.User shadowing: This is a qualitative data collection technique where the behaviour and actions of the user are observed and recorded and is further used for analysis.
The emotions of the user are recorded as well. The ethic rule is that observation recordings of the patient are saved in a device that requires password to access it so that the data stored cannot be used by external sources or cannot be accessed easily. Focus Groups:Focus Groups are similar to interviews but are more effective since many people can be interviewed at once. Conducted during data collection stage of a product. Ethical rules: The Focus group recordings will be saved and handled with care Diary studies:Dairy study is used to collect qualitative data about user behaviors, activities, and experiences over time.During the training and briefing sessions the patients are asked to follow a specific scheme of recording data where no other people can understand the data without proper knowledge how the data is encrypted.Data Logging: Data loggers are the softwares used to input data inside the system and store them for later user.
This can be manual as well as automated. A coding scheme is pre established in the system so that the data are always encrypted and stored.In general if the place of where the data is stored and what security measures that the device which contains the data possess and who have access to those devices and data are secured then the data will possess confidentiality. Loss of data is also a factor of less safety of data so necessary measures should be taken to backup data properly. Data circulation channels must be secured. ANSWER 2:1.Focus group together with diary study: A focus group is a method of data collection which involves group interaction managed by a moderator in an interactive environment. Dairy study is used to gather behaviors, activities, and experiences of user over time.
Pros: Focus group(FG) is cost effective and easy to conduct.FG generates opportunity to collect data from the group interaction.Immediate results can be identified through FG method. FG allows a relatively large sample size for a qualitative study.
Diary study(DS) helps to understand temporal dynamics. DS also helps to get data related to events and experiences in specific context. Cons: FG – Not very suitable for testing usability because multiple people use software at the same time and observing becomes difficult.
Difficulty in analyzing data. Assembling a group may require additional resources. DS- Finding the target group for DS is difficult.
Data analysis is hard.Training and briefing sessions for DS takes more time.2.Field evaluation methods(in the wild): This is evaluating the usability of the system in a natural setting. It is uncontrolled and thus allows more usability freedom for the users. Pros: Helps to identify problems. Finds external sources that affects the usability of the system or product since the environment is uncontrolled like home or workplace and not lab controlled.
Cons: More distractions and loss of data quality and poor observations.3.Cooperative evaluation study: Cooperative evaluation is a kind of think aloud method, in which the user acts as a collaborator in the evaluation rather than just a subject. Pros: Less constrained and easier to use. User is encouraged to criticize system. More clarification is possible. Cons: Ends up in large data and analysing it is tough. Involves protocol analysis which has to be done by hand.
Considering the pros and cons of all the three methods above and considering the needs of the customer, field usability evaluation method is more suitable fulfilling the needs of the customer more precise. And the method is more context related and takes less time than focus group with dairy study and will be more efficient than the cooperative evaluation study. The test plan is as follows: Purpose: To evaluate usability of the interaction flow between Wellmo apps, services and devices and to understand the learnability of the and safety of them. To generate insights that will be used for future development of the system. Participant characteristics: People from wellness service providers and employers. Method: Field Usability Evaluation based on Semi structured interviews, tasks evaluation, audio and video recording. Task list: Tasks that involves testing the interaction and identifying the learnability and safety of the wellmo ecosystem.
Test environment: User’s environment.Dependent variables : 1.Speed (time to complete all of the tasks),2.
Number of interactions made to complete task. 3.Completion of task steps (success/failure), 4.
Errors while doing a task, 5. Errors which leads to less safety. Independent variables: 1.User location,2.Iteration of task,3.Task step. Coding scheme will be done based on the data required for analysis.
Since the objective focuses on interaction flow and learnability and safety the coding scheme will have the dependent variables mentioned here. The report will contain the statistical analysis of the test which provides insights about the wellmo apps and services ecosystem and how the system can be improved further. The report will also express how learnable and safe the application is. Some suggestions to improve the drawbacks that will be found will also be provided. Question 3:Test1: Exploratory/Comparison Test:Objectives and Research Questions:Exploratory tests are those done when a design of a product or system is being made. This occurs during the early development stage of a product. This helps to understand the how well the design meets the initial features of the personal information manager software.This early testing method will help to identify potential factors like how the user understands the design of the personal information manager software(PIMS)?, The actual user experience of the product?, To understand whether a user can be able to perform task without seeking help documentation?, To know what necessary information are needed to the user to perform basic tasks? and many more.
Comparison tests are those can be used in conjunction with the above tests. It is used to compare two or more designs of the product to understand which is more usable. Since the PIMS has two prototypes, similar tasks are made to perform on both the designs and respective data are collected which will eventually lead to discard the least scoring prototype.
Helps in bringing out more creative design features to the system.Data collected(Both):Data collected from exploratory/conservative study will be based on the system or product being tested. Considering testing of “personal information manager” through exploratory study the data will be qualitative since exploratory study is more of thought process of the user and conceptual understanding of them. The data can be in the form of answers made for the “HOW” questions posed by the test examiner. Conservative study will lead to quantitative data since it compares two or more system and its usability. Some data that we can focus during conservative study are which model is easy to use?, Speed of each task?, Completion of each task?, steps take to complete each task?, Learnability of the application? and many more between the two prototypes are collected. Here data for different iterations are also collected to understand better usability and learnability of the two systems.Data Analysis:Qualitative and quantitative data analysis both requires coding of the data gathered depending on the need of the data for analysis and this “personal information manager” requires both kind of data to be coded.
A consistent code base should be created to code the data. The coding of data can be done using softwares like Observer XT,Boris,Maxqda etc. Interviews are transcribed and common themes are identified and they are taken into account for analysis.Dependent and Independent variables are identified for better analysis. The coded data are then gathered and grouped and are analysed based on various descriptive statistical measures like looking at the sum values, mean values and the ratio between different values.Data is also analysed for One tailed and two tailed T – tests.Test 2: Assessment Test: Objectives and Research Questions:Assessment type tests that are carried out during the mid stage development of the produce and it tries to address the issues raised in Exploratory/Comparison testing.
The user performs test using think aloud based and data is gathered from it. Testing the “Personal Information Manager” at this stage is to test the application in real time and to find the effectiveness, satisfaction, further understanding and overall usability of the web application. All the functionalities that are developed till date are tested for its working and its usability and learnability. It answers questions like how strong exploratory study is incorporated into the system.
It also focuses on user experience while using the application and further changes that should be made to the system. Data collected(Quantitative):A set of tasks are asked to perform by the user and the time taken to complete tasks, and completion rate, error rate, ability of user to finish tasks without user manual, ability to use user manual etc etc are coded. Number of time the user referred to the help documentation is also collected. Data are collected for different iterations.
Data Analysis:Data collected are first coded using softwares like Observer XT,Boris,Maxqda etc.The coded data are then gathered and grouped and are analysed based on various descriptive statistical measures like looking at the sum values, mean values and the ratio between different values.Data is also analysed for One tailed and two tailed T – tests.Test 3: Verification test:Objectives and Research Questions: This test is carried out after a product is fully developed and is ready for testing before its release. The application is tested to pass some prespecified standards and benchmarks. In this phase the personal information manager is tested for its full functionality without previous errors and new erros and its ability to meet the specified measurable time, benchmarks and accuracy criteria developed. The question that it addresses is whether the whole product is usable and behaves as per the accuracy criteria fixed and further standardized measures.
Data collected(More Quantitative): The data that can be collected from here can be equal as those collected during the assessment test and apart from that data like the system’s ability to pass(yes/no) the fixed criteria to execute certain task is collected. Data for old and new error are collected.Data analysis: Data are analysed as same as in the assessment test for understanding usability and learnability of the application. The data collected to verify the criteria or measurement time to complete a task are analysed in a way such that whether it succeeds or fails. Data is also analysed for One tailed and two tailed T – tests.various descriptive statistical measures like looking at the sum values, mean values and the ratio between different values.Data is also analysed for One tailed and two tailed T – tests are done.
ANSWER 4:Feedback:The need of open interview to selection candidates for both comparison groups can be avoided since you already recruit users based on questionnaire and simple tasks on MS word processor so that the hiring of users will be effective and further segmented based on the needs of the focus group.Demographics of participants are needed.If pre task is not going to be used for analysis it can be avoided to save time.
Independant/Dependant variable:In dependant variable the number of links/actions can be separated since the number of actions corresponds to the clicks or key press made to perform the task and the number of links refers to the links provided in the help menu by the processor which in some cases can be default number. Tasks should move from independent variable to dependant variable since it affects the number of actions or number of links that a user has to proceed because of the difficulty of the tasks.Each help state “Contents”, “Answer Wizard” and “Index” should be considered inside dependant variables since it will affect the steps taken to reach a solution.Other:Error events like mistyping or system error has no effect on the feature “word help” which is being tested, it can be omitted.Develop a common coding scheme to record the data for further analysis with all the state events,point events.
Post test questionnaires can also include how difficult the tasks were and whether they can do the same tasks with more ease again?Information about what the test participant should do must be clear. In scenario 1 it is expected to have one task but it contains 3 sub tasks in it which are counted as different tasks but comes under single scenario. How this scenario 1 data will be collected and analyzed? It is suggested to split the sub tasks in scenario 1 into separate tasks so that proper understanding of tasks and data collection can be achievable.