we invited one of Denmarks leading AI and data analytics consultancy companies, Damvad Analytics, to validate and further refine our ability to use AI to automatically generate unit descriptions in courses based on data dumps from the course creator.
Starting in May 2021 we took part in an AI innovation project supported by the Danish Industry Foundation, where we invited one of Denmarks leading AI and data analytics consultancy companies, Damvad Analytics, to validate and further refine our ability to use AI to automatically generate unit descriptions in courses based on data dumps from the course creator. For the external audit, the following sampling was used:
Total number of courses: 101
Total number of units: 314
Total number of materials: 768
The courses were in English – though some material included text in other languages. For now, all AI features are available in English only.
In the workshop, which culminated their analysis, Damvad Analytics raised the relevant question: What constitutes a great summary? To improve upon the autogenerated summaries, we need to agree on what a good summary is. Damvad Analytics concluded:
- A summary/description of the unit should be short and concise
- To measure the quality of a summary two aspects are needed.
- Qualitative: Does the summary read well, and does it cover the topics of
the unit? - Quantitative: How long and complex is the summary?
- Qualitative: Does the summary read well, and does it cover the topics of
the AI auto-generated descriptions were longer and more complex than the human-written summaries across our learning platforms. To further improve upon the summaries, they recommended that we switch from the popular BERT model to BART as “BART improves summaries by generating new sentences which are easier to read for humans” according to Damvad
Damvad Analysis indicated that the AI auto-generated descriptions were longer and more complex than the human-written summaries across our learning platforms. To further improve upon the summaries, they recommended that we switch from the popular BERT model to BART as “BART improves summaries by generating new sentences which are easier to read for humans” according to Damvad.
In a recent Linkedin post Damvad comments on our work with AI in stating:
A pleasant learning experience makes learning a little easier. A good introduction to learning material makes the learning experience more pleasant and reduces the learning barrier. CanopyLab uses Artificial Intelligence (AI) to extract important sentences from learning material to create a summary of the learning material. These can serve as an introduction to the learning material. But it turns out the most important sentences are rarely the best summaries, and they can be difficult to understand without the context.
Our Head of Artificial Intelligence Tracy Tran reflects on the joint AI improvement project: We had the most experience working with BERT, but in our collaboration with Damvad Analytics we gained many valuable insights as they’ve done a more detailed evaluation and comparison between different AI methods and models across Industrie. We actively leveraged their experience of implementing and evaluating different methods and used them to brainstorm new approaches. Finally, they offered us an understanding of how our AI measures up compared to other platforms.
Damvad Analytics concluded the following about their contribution to our new and improved model: “By leveraging recent advances in language models for natural language processing, Damvad Analytics demonstrated that it is possible to generate short, concise summaries of the teaching material. These summaries are designed to reflect the key concepts of the material with surprising eloquence. They are easy to read and can be auto-generated from all types of material, including videos, articles, and web pages. A pleasant learning experience makes it easier to learn. The auto-generated summaries of teaching material can help students focus on what is important. It also helps educators spend more time teaching and less time writing. This means more learning, with the help of Artificial Intelligence”.
Our new and improved AI feature elevates the learning experience and we look forward to pushing the boundaries of what is possible with NLP in learning even further.
