Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Sv translation
languageen

Table of Contents

Current QA activities in this column

Sakai 19 QA Hub

QA Servers

Meeting Logistics:



General Information about QA in this column

Purpose

QA stands for Quality Assurance. In this context the QA mission is primarily to uncover and report software bugs, to verify fixes to software bugs, and to test for regressions.  Regressions are features that worked in previous versions, but are broken in the current version.

Pre-requisites for participating

Required - a Jira account is required. Sign up

at

at https://jira.sakaiproject.org

VideoHow to Create a Jira (3:26 minutes)

Required - Join Sakai QA email group (sakai-qa@apereo.org) or Sakai Dev group (sakai-dev@apereo.org) to keep up to date with the latest announcements and ways to help with the testing effort.

Optional - Experienced Testers - Although, you do not need special permissions to test fixes and to comment on Jira issues, you can not mark the Jira as having its testing completed. To mark the Jira as completed, you need access to a "Tested" button which indicates successful testing of a Jira issue. This button is only available if you are a member of the Jira QA group. Contact sakaicoordinator@apereo.org for more information.

Hints - Our primary tools are Jira, Google docs spreadsheets, and QA servers. Learning how to use Jira and what the fields represent is well worth the time.

Ways to Participate

  • Verify bugs that have been fixed.
  • Test new features.
  • Test existing features to assure changes around them haven't introduced bugs (regressions).
  • Create Regression scripts (i.e. step-by-step instructions for performing regression tests.)
  • Release testing - testing alpha, beta, and release candidate versions before software is made generally available (GA aka "General Availability" aka production ready). 
  • Proofread. Make suggestions. Create short videos. 
  • Ask and answer questions on the email groups. 
  • Attend the QA meetings to bring up issues that need attention, and to help plan testing for releases.
  • Expert QA knowledge needed - special skills are needed for some types of QA testing.

When to Participate

for
  •  for help getting started.
  • Major Releases - Sakai 11, Sakai 12 and Sakai 19 are all examples of major releases. Major releases typically include significant upgrades to existing features and the addition of new features.
  • Maintenance Releases - Sakai 11.4 and Sakai 12.6 are examples of maintenance releases. 

Anchor
VerifyingBugs
VerifyingBugs
Verifying bug fixes

Testing either takes place on a nightly server, usually trunk, , or on a QA server that is announced when community testing for major or maintenance releases is scheduled. Please be aware that nightly servers are refreshed daily, at midnight Eastern time (-4 or -5 GMT). Please plan your testing appropriately. 


Jira bugs which have a resolution of "Fixed" and a status of "Resolved" are ready for testing. If testing is successful click the Tested button. The status will change to Verified. If you don't see a Tested button, even with the correct status and resolution, you may not have permissions in Jira to see the button. Contact sakaicoordinator@apereo.org to request this permission.

Image Modified

In your comments make sure to include which OS/browsers you tested and the Revision number of trunk. The Revision number is important because trunk changes frequently. Look at the bottom of your browser window for the Revision number. 

Image Modified

Notes: Some issues may require testing in more than one OS /Browser combination. 

Special Case: Master (formerly trunk vs Branch)

The Master branch (previously known as Trunk) has the most recent source code, and is constantly changing as developers contribute fixes and new features. At some point, when we are getting ready to release Sakai as a community-supported version (aka general availability or production release), we create a branch. A branch is a copy of master at a certain point in time, a snapshot. By doing this, we can exercise more control over what goes into a release. We name a branch to correspond with the release. For Sakai 12.0 release we have a 12.0 branch, which we sometimes also refer to as 12.x. For Sakai 19.0 release we have a 19 branch, also called 19.x .  Features and fixes almost always start in master (more than 99% of the time. Exceptions are rare.) We test features in master and then when they are verified to work, if they should be included in a branch, we merge them into the branch and re-test the fixes, since the environment between master and the branches can diverge significantly. A fix that works in master, may not work in the branch (although in most cases it does.) Since we do not have enough QA resources to retest every fix, we focus on getting at least all the blocker priority issues retested.

We include in JIRA a status that indicates the state of the ticket with respect to the branch. For example: 12 status. When it is set to Resolved it means that the fix has been merged into the Sakai 12.x branch. Verified means that it has been tested after merging into 12.x . This can be confusing, until you get used to it, because it means that there are at least two fields on a JIRA ticket that have the potential to be set as Resolved or Verified, the main Status of the ticket (see image above), and the branch status. There are only 2 versions of Sakai in development at any one time. Currently that is 12 and 19, therefore we have a 12 status field and 19 status field. When Sakai 19 is released, we will drop support for Sakai 11 and there will be an 19 status field and a 20 status field.

As a QA tester, once you understand what the fields mean, it is okay to set the status yourself to Verified, on both the overall JIRA and for the branch.


Image Modified

A bug that is supposed to be fixed, isn't

If you are verifying a fix and it fails then reopen the issue by clicking the Reopen button. This indicates to the developer that there is more work to be done. If you are not sure if the issue should be reopened then simply leave a comment. Make sure to include the exact steps of how it failed, what server you were on, the revision number if testing was done on trunk, browser/os, etc.

Anchor
TestNewFeatures
TestNewFeatures
Test New Features

Testing new features is essentially the same as testing for bug fixes. You still want to click the Tested button to move the issue to Verified status if all the testing passes, and Reopen or add comments if you find bugs. Sometimes there are debates about new features and what constitutes a "bug". Those debates happen in the Jira comments and sometimes are brought to other groups like the Sakai Core team for broader discussion, as appropriate/relevant. 

Anchor
RegressionTest
RegressionTest
Regression testing

Regression testing is a kind of detailed testing. Regressions are features that worked in previous versions of Sakai, but now are broken in the current version. It It is very important to uncover these kinds of problems as soon as possible. 

The most up-to-date list of tools for which we have Regression scripts (steps for testing features manually) is kept in Apereo's Google drive. The most recent set is

for

for Sakai 19 RC.1

Sakai 11 Scripts-do we have updated copies?

Release testing - QA Test Hubs

Our goal is to have testing happen as much as possible throughout the Sakai version lifecycle (verifying bug fixes in trunk, testing new features, and regression testing the latest release). However, when we are getting close to having a tagged release (e.g. 12.6, 19.0, 19.1RC, etc) we choose specific periods of time to test, with a Test plan and focus. We call these testing efforts "Test Fests". We have multiple Test Fests to get out a release. Unless otherwise specified we have one "Test Fest" per phase of the release. For example, we might have several Beta versions of the software before the Sakai core team feels confident that it is ready for release. Typically, we then have two or three Release Candidates of the software. A Release Candidate is a version of the software, which if it passes QA testing and Developer scrutiny, will become the next official community release. Each of these Beta releases and Release candidates is a phase of our Software Release process. 

What makes a Test Fest unique is that we create a Test plan to guide our testing, a defined period of time to complete the testing, and a review process to assess the state of our testing results. Test Fest's typically take a concentrated effort and we like to have as many testers as possible helping in the effort, sometimes with multiple testers focusing on the same functional areas to provide the highest confidence level we can reasonably achieve. Test Fests are almost always carried out

on which

 which have been updated with the correct version of the software which needs testing. The Test Plan will indicate which QA servers are available for testing. 

Anchor
QATeamMeetings
QATeamMeetings
QA team meetings

QA Planning Meetings Tuesdays 2 pm EST BigBlueButton Sakai QA Room. 

QA Test Fest Meetings Thursdays 10 AM EST BigBlueButton Sakai QA Room  

Anchor
ExpertQAKnowledge
ExpertQAKnowledge
Expert QA knowledge

  • Accessibility Testing
  • Localization Testing (different languages)
  • Review of patches and fixes
  • Testing locally
  • Automated testing
  • Security testing (requires special permission)

General Testing Tips

Include Page
General Testing Tips
General Testing Tips

Current QA activities in this column

Sakai 19 QA Hub

QA Servers

Meeting Logistics:

Sv translation
languagees

Table of Contents

Propósito

QA significa Aseguramiento de la Calidad. En este contexto, la misión de la QA es principalmente descubrir y reportar errores en el software, verificar los arreglos de estos errores y comprobar posibles regresiones. Las regresiones son funcionalidades que funcionaron en versiones anteriores pero están rotas en la versión actual.

Pre-requisitos para participar

Requerido - Se requiere una cuenta en Jira. Inscríbase en https://jira.sakaiproject.org

RequeridoUnase a la lista de correo Sakai QA (sakai-qa@collab.sakaiproject.org) o a Sakai Dev (sakai-dev@collab.sakaiproject.org) para estar informado de los últimos anuncios y de las distintas formas de ayudar en las iniciativas comunes de testeo.

Opcional - Grupo JIRA QA - permisos especiales - No son necesarios permisos especiales para testear arreglos y para comentar en los tickets de Jira. En JIRA hay un botón "Tested" que indica el testeo con éxito del ticket. Este botón sólo está disponible para los miembros del grupo Jira QA. El permiso se proporciona a testers experimentados. Contacte con neal.caidin@apereo.org para más información.

Trucos - Nuestras principales herramientas son JIRA, hojas de cálculo de Google Docs y servidores QA. Aprender cómo usar JIRA y qué función tiene cada campo de un ticket merece la pena.

Formas de participar

  • Verificando errores que han sido arreglados.
  • Probando nuevas funcionalidades.
  • Test de regresión de funcionalidades existentes.
  • Creando scripts de Regresión (p.ej. instrucciones paso a paso para comprobar una regresión).
  • Testeo de Releases - Probar versiones alpha, beta y RC (Candidatas para ser Releases) antes de que el software esté disponible públicamente (donde "Disponible públicamente" significa "Listo para ser usado en producción").
  • Ayudando a mantener la documentación de QA en Confluence. Correcciones. Sugerencias. Creación de vídeos cortos.
  • Preguntando y respondiendo a preguntas en las listas de correo.
  • Reuniones con el equipo QA. Acudir a las reuniones QA para discutir de problemas que requieran atención, para ayudar a planificar el testeo de releases, etc.
  • Conocimiento experto de QA necesario - Algunos tipos de QA necesitan de habilidades especiales por parte del testeador.

Anchor
VerifyingBugs
VerifyingBugs
Verificando errores arreglados

La mayor parte del testeo debería hacerse en el servidor Trunk QA nightly server. Debe tenerse en cuenta que este servidor se actualiza a la última versión de Sakai cada 4 horas, comenzando cada medianoche en horario Eastern (-4 o -5 GMT). Por ello, debe planificarse el testeo apropiadamente. Cuesta unos 20 minutos que Sakai se recompile y se despliegue. Así que, por ejemplo, entre las 12:20 pm y las 3:59 pm (en horario Eastern) sería una buena hora para trabajar en él.

Los tickets de JIRA que tienen "Fixed" en el campo resolution y "Resolved" en el campo status están listos para ser testeados. Si el testeo es exitoso se debe pulsar el botón Tested, tras lo que el status cambiará a "Verified". Si el botón Tested no es visible, incluso con los campos status y resolution correctos, es probable que se deba a una falta de permisos para ver dicho botón. Es necesario contactar con neal.caidin@apereo.org para solicitar este permiso.

Al documentar un ticket, se deben incluir los navegadores y sistemas operativos en los que se ha producido el testeo y el número de revisión de la versión trunk. El número de Revisión es importante porque el código de la versión trunk cambia frecuentemente. El número de revisión aparece en el fondo de la ventana del navegador.

Nota: Algunos problemas pueden requerir testeo en más de una combinación de S.O. y navegador.

Un error que supuestamente está arreglado, ¿no?

Si se verifica un arreglo y no funciona, entonces debe reabrirse el ticket con el botón Reopen. Esto indica al desarrollador que falta trabajo por hacer. Si no se está seguro de si el ticket debería ser reabierto, puede ponerse un comentario para que otros lo decidan. En el comentario se deben incluir los pasos exactos de cómo falló, en qué servidor, el número de revisión de dicho servidor, navegador/S.O., etc.

Anchor
TestNewFeatures
TestNewFeatures
Pruebas de Nuevas Funcionalidades

Probar nuevas funcionalidades funciona básicamente igual que probar arreglos de errores. También se pulsa el botón Tested para que el ticket pase a estar con estatus Verified si el testeo es correcto y el botón Reopen (o el comentario oportuno) si no lo es. A veces sucede que se debate sobre si un ticket es un error o una nueva funcionalidad. Estos debates suceden en los comentarios del ticket y a veces se trasladan a grupos como el Sakai Core Team para una discusión más amplia.

Anchor
RegressionTest
RegressionTest
Testeo de Regresiones

El testeo de regresiones consiste en probar al detalle una herramienta comprobando si tiene "regresiones". Las regresiones son funcionalidades de la herramienta que funcionaban en versiones anteriores de Sakai pero que no lo hacen en la versión actual. El testeo de regresiones busca descubrir estos fallos, repitiendo manualmente una serie de pasos en la herramienta que deben dar un determinado resultado.

La lista más actualizada de herramientas para las que existen scripts de regresiones (pasos para comprobar manualmente las funcionalidades) y de sus scripts se almacena en un documento Tools Google doc.

Testeo de Releases - Centros de Pruebas QA

El objetivo principal es tener el máximo testeo posible a lo largo del ciclo de vida de una versión de Sakai (verificar arreglos de errores en trunk, testear nuevas funcionalidades y testeo de regresiones en la versión más actual). Sin embargo, cuando se acerca el momento de que se lance una release etiquetada (p. ej. 2.9.0, 2.9.1, 2.9.2, 2.9.3, etc) se planifican periodos de tiempo específicos para testear, utilizando un Plan de Testeo y concentrando a los testeadores en él. A estos esfuerzos colaborativos se les llama "Test Fests".

Hay multiples Test Fests en una Release. A menos que se especifique lo contrario, hay una Test Fest por cada fase de la Release. Por ejemplo, podrían publicarse varias versiones Beta del software antes de que el Sakai Core Team apruebe que dicho software está lo suficientemente maduro.  Tras ello, habitualmente se sacan dos o tres versiones Release Candidate del software. Una versión Release Candidate es una versión del software que, si pasa el testeo QA y el escrutinio de los desarrolladores, se convertirá en la siguiente versión etiquetada de la Comunidad Sakai. Cada versión Beta y RC es una fase del proceso de Publicación de Software del Proyecto Sakai.

Lo que hace especial una Test Fest es que se crea un Plan de Testeo que guía a los testeadores, un periodo de tiempo definido para completar el testeo y un proceso de revisión que evalúa los resultados del testeo. Las Test Fests normalmente concentran los esfuerzos y necesitan de la ayuda de tantos testeadores como sea posible, a veces con muchos testeadores enfocándose en la misma área funcional, de forma que se alcance el máximo nivel de confianza posible sobre el testeo. Las Test Fests casi siempre son celebradas en los servidores QA especiales que son actualizados a la versión correcta del software que se pretende testear. El Plan de Testeo indicará qué servidores QA están disponibles para trabajar en cada momento.

Anchor
QATeamMeetings
QATeamMeetings
Reuniones del equipo QA

Se debe estar atento a la lista de correo. Más información sobre este tema será proporcionada.

Anchor
ExpertQAKnowledge
ExpertQAKnowledge
Conocimiento Experto de la QA

  • Testeo de accesibilidad
  • Testeo de la Localización (diferentes lenguajes)
  • Revisión de parches y arreglos
  • Testeo local
  • Testeo automático
  • Testeo de seguridad (requiere un permiso especial)

Consejos Generales de Testeo

Include Page
General Testing Tips
General Testing Tips