Q4R Quality Assurance Workflow Model

Download the complete AGDI Model

This model stems from examining about 300 repositories and having 22 of them answering a questionnaire eliciting information about what kinds of quality assurance they use at the organizational level, such as the use of policies, membership to control contribution and at the daily use level, such as incorporating different types of feedback systems, such as peer-reviews of LO, social tagging, rating systems and annotations.

In general, either a demand is issued within the organization to create a LOR for its members or a proposition is received from a bigger network of repositories to individual institutions to join in the effort to build institutional, nationwide or worldwide repositories. Examples of such networks are Simon Fraser Institutional Repository (http://ir.lib.sfu.ca/ ), the European SchoolNet (http://www.eun.org/portal/index.htm ), Education Network Australia (http://www.edna.edu.au/edna/go ) and  GLOBE Network.

The model basically consists of 5 main procedures, three describing the establishment and maintenance of the repository, and two describing the continuous flow of feedback from all the actors. The three first procedures treat:

• Needs analysis, whether it be according to initial demands and propositions or according to revision demanded according feedback received while in use.
• Elaboration of the Q4R Quality Assurance KIT
• Use and Maintain Quality in the LOR

These three procedures will be described in further detail below.

Analyse Organizational Needs

Analyzing needs is a continuous activity and is triggered either by a proposition from a larger network of repositories or by internal demands. As examples of the former type of proposition, the following can be mentioned:  an institution asking all its departments to use a repository (Simon Fraser) or a nationwide attempt (European SchoolNet; Education Australia) to provide a community of users to join in an share resources or a worldwide organization trying to open sharing of resources regardless of country and cultural diversity (GLOBE).

Internal demands often springs from teachers and / or students asking for or suggesting a Repository but also from ‘champions’ or training managers suggesting using a repository for teachers and learners.

A very helpful site is The Repositories Support Project (RSP), a major JISC initiative to support the development and growth of the UK repositories network (http://www.rsp.ac.uk/repos/scope). The table below describes main activities, rules of thumb, who should be responsible, resources needed to carry out the activity and what the activity will produce (deliverables).

Activity	Responsible	Rule of thumb	Input Resources	Products
Analyze demands for LOR	Manager	Involve as many stakeholders as possible	Network proposition Request for LOR within the organization	Decision to incorporate a LOR into the organization.
Formulate LOR’S goal and objectives	Manager	It is essential to formulate goals and objectives in order to ensure a common understanding of what the LOR is to accomplish	Examples of goal, mission and vision statements	LOR’s list of goals and objectives Mission statement Vision statement
Identify potential end-users and needs	Management team	List all potential end users and their particular needs
Identify type of LO’s  to be integrated	Technical staff in collaboration with pertinent end-users	It is important to define the type of LO’s that the LOR will host, because it might demand different types of technical infrastructures.	Example resources, such as i.e., SCORM packages, video, pictures, etc. all kinds	List of LO’s Compatibility Issues
Identify Repository Communication Channels	Technical staff in collaboration with pertinent end-users	Good communication channels reinforces quality.	For ideas see Q4R Website under the Worldwide Repository page	A choice of Feed back systems • Contact persons • Information • Events Calendar • New resources display system • Newsletter • Award events
Identify and list constraints	Technical staff in collaboration with pertinent end-users	List joint projects influencing the organizational and / or technical infrastructure needed.	For example, there is a demand for SCORM compliant learning objects and then it is important to provide scorm compliant production and validation software	List of tools pertinent to the decisions in the organization

Elaborate the Q4R KIT

A Q4R Kit is the set of policies, guides, tutorials, tools and job aids that are chosen in order to develop and maintain quality in a Repository that is to ensure that reuse is high.  Efforts that aim at developing and maintaining the quality of a LOR can not ignore the importance of a critical mass of users who are ready to both contribute and reuse the learning objects. To reach this tipping point, or critical mass, that is to ensure reusability, the following points can be considered as important:

• Elaborate change management strategies
• Adopt moderate quality assurance strategies
• Include automatic feedback systems
• Include automatic metadata generation
• Provide easy to understand and applied copyright statements

The main decisions here are to investigate whether it is beneficial for the community to join an existing network of repositories.

Activity	Responsible	Rule of thumb	Input Resources	Products
Join Network? 1. Answer the question whether to join an existing LOR Network is desirable or not. If yes, find out if they impose Q4R KIT and whether your organization can adopt it.	LOR Management Team	Investigate pros and cons in joining an existing LOR network in the light of needs and constraints, goals and objectives.	• List of needs and constraints • Mission and vision statements • List of goals and objectives • GLOBAL • European SchoolNet • Eurêka	Decision on whether to build a new LOR or not.
• If the Q4R KIT is acceptable, start using the LOR.			Use and Maintain LOR
Build New LOR?		If yes, Create a LOR Architecture Team	Building Repositories	• Technical Infrastructure • Interface functionalities
Select quality assurance elements desired in your organization.		Policies are essential to high quality, since these put forth criteria for different aspects of quality.	• LOR Management Team Job Aid • Q4R Repository	A list of • Policies • Job Aids • Guides • Tutorials That must be developed
Elaborate desired policies		It essential to create mixed teams including pedagogical, technical and administrative staff.		Documented and ready to use Policies
Adapt and elaborate job aids for each role			Job aid template, or examples of job aids	Adapted job aids for each role

Use and Maintain LOR Quality

This model shows the main activities that each role carry out and how it creates a high quality LOR, may it be a Referatory (stores metadata records and a url for access to the object itself) or a Repository (stores the object + the metadata record).

The Q4R Feedback Loop to Assure Quality

The feedback loop implies that all users of the LOR participate in evaluating and maintaining the quality of the LOR and its learning objects. To do so the policies provide guidelines on how the repository should work and what is important. It is therefore important that these are respected. In terms of technical quality it is also essential that all users regardless of role played in the system, reports any difficulties or malfunctioning. It is therefore important that the LOR Management Team provides easy to use feedback systems. For example, if a LO is not in working order or a link is broken, this should be automatically reported, but if it breaks after use, it must be reported by the Reuser.

Evaluate LOR quality and report

All actor roles are implicated in the maintenance of the quality of the LOR. They do so by participating in the elaboration of the policies and by respecting these types of quality assurance measures. The policies and metadata guidelines are documents that provide evaluation criteria.

Pedagogical
Technical
Socio-Cultural
Accessibility and ergonomics

Examine feedback and list revisions

The LOR Management team is responsible for reviewing at a regular basis the feedback coming in and distributing revision elements to pertinent persons in the organization.  This is a continuous activity that helps maintaining quality in the Repository.

NEXT: Glossary of terms