Ministry of Education, Culture, Sports, Science and Technology Commissioned Project for University Entrance Examination Reform Promotion(Utilization of CBT in Individual Universities’ Entrance Examinations and Related Selection Processes)
Project Title
A Trial of University Entrance Examinations Using Item-Bank-Based CBT Including “Informatics I”
Project Overview
This project aims to build item banks (question banks) for “Informatics,” “Mathematics,” “Physics,” and “Non-Cognitive Abilities” at The University of Electro-Communications, and to conduct entrance examinations for the 2024 academic year using CBT (Computer Based Testing) in both the school-recommendation-based selection and the comprehensive selection. The main features of this project are as follows. (1) In accordance with international standards for the implementation and operation of CBT (ISO/IEC 2007, JIS X 7221), the project adopts an item-bank method using IRT (Item Response Theory). It develops an advanced AI-equipped system with test assembly technology that achieves the world’s highest level of measurement accuracy. (2) For the first time in Japan, CBT entrance examinations including “Informatics I” are conducted in the comprehensive selection and school-recommendation-based selection. The system evaluates practical abilities that could not be measured by conventional paper-based or oral examinations alone, such as questions in which examinees edit and execute programs in an actual programming environment to derive answers, and questions in which examinees analyze real data using data analysis tools to derive answers. (3) For programming and data analysis questions, examinees’ trial-and-error answer processes, such as program execution and displayed errors, are also recorded and can be taken into account in automated scoring.
Furthermore, through this project, we also aim to accumulate knowledge on the use of CBT in small-scale examinations and related settings in order to address the issues identified in the National Center for University Entrance Examinations report “Possibilities for the Use of CBT in Large-Scale University Entrance Examinations” (March 24, 2021): (1) securing a uniform and high-quality testing environment, (2) establishing a response system for handling problems, and (3) fostering understanding among examinees and society as a whole regarding new forms of examinations.
Implementation Organization
Progress to Date
- October 2022: Project commissioned
- October–December 2022: Development of item-writing procedures (creation of an item-writing manual and question checklist)
- October 2022–March 2023: Creation of questions for mathematics, physics, informatics, and non-cognitive ability surveys
- October 2022–March 2023: Development of a prototype CBT system
- December 2022: External Evaluation Committee meeting held
- February 2023: University-wide internal meeting held
- March 2023: Information gathering at the National Convention of the Information Processing Society of Japan
- April–October 2023: Conducted a CBT-based basic academic skills survey (mathematics, physics, and informatics) for first-year students at UEC
- April–October 2023: Analysis and compilation of results from the CBT examination (UEC Proficiency Test)
- April 2023–March 2024: Continued creation of questions for mathematics, physics, informatics, and non-cognitive ability surveys
- June 2023–March 2024: Promotion of inter-university collaboration toward the use of CBT in university entrance examinations
- November 2023: CBT trial session and media briefing held at The University of Electro-Communications Open Campus (Link）
- Conducted a CBT trial session using sample questions for second-year high school students nationwide
- Held a briefing for newspapers and other media on the implementation policy for CBT entrance examinations (NHK coverage）
- December 2023: Sample programming and data science questions released on The University of Electro-Communications website (Link）
- December 2023: University-wide internal meeting held
- January 2024: External Evaluation Committee meeting held
- April 2024: Conducted a CBT-based basic academic skills survey (mathematics, physics, and informatics) for first-year students at UEC
- May 2024: Briefing and CBT trial session for high school teachers on CBT entrance examinations (Link）
- June 2024: Agreement concluded on collaboration for promoting the use of CBT in university entrance examinations (Link）
- July 2024: CBT trial session for high school students held at the Open Campus (Link）
- September 2024: CBT pre-trial environment made available (available for a limited period: September and November)
- September 2024: CBT conducted in the comprehensive selection (Cluster I) (mathematics, informatics, and non-cognitive ability survey)
- November 2024: CBT conducted in the school-recommendation-based selection (Cluster I) (mathematics, informatics, and non-cognitive ability survey)
- December 2024: CBT Project Report Symposium and External Evaluation Committee meeting held
Basic Explanation of IRT (Item Response Theory) (Video)
Public Information
-
- September 2023: The University of Electro-Communications “Briefing Session for High School Teachers on the 2022 Academic Year Admissions Selection” (Link）
-
- November 2023: NHK coverage
-
- December 2023: Article in The Nikkei, “Will Computer-Based Entrance Exams Spread? A Fresh Start for Adoption in the First Year of ‘Informatics’” (Members-only link）
-
- December 2023: Grant-in-Aid for Scientific Research (S) Symposium, “CBT Practice at UEC”
-
- March 2024: Article in Gakken Shingaku Joho, “Special Feature: The Potential of CBT in Admissions Selection — Measuring Thinking and Creativity That Paper Tests Cannot: The University of Electro-Communications’ CBT Entrance Examination” (Link）
-
- June 2024: New Education Expo 2024 seminar presentation, “Can CBT Change Entrance Examinations? A National University Entrance Examination Project Using AI and World-Class Technology” (Link）
-
- July 2024: Article in The Asahi Shimbun, “Predicting Post-Admission Grades and Failing Grades: Expanding Use of Computers in Entrance Exams and Academic Skills Surveys” (Paid article link）
-
- August 2024: Kickoff Symposium for the CBT Utilization Collaboration Agreement, “The University of Electro-Communications’ CBT Entrance Examination Meeting Global Standards” (Link）
-
- November 2024: Article in Chofu Keizai Shimbun, “Computer-Based Entrance Examination Symposium at UEC in Chofu; Prospective Students and Parents Also Invited” (Link）
-
- March 2026: MEXT Commissioned Project for Promoting Reform of University Entrance Examinations (Use of CBT in Individual Universities’ Admissions Selection, etc.) (FY 2022–FY 2024)
Final Report (Link)
- March 2026: MEXT Commissioned Project for Promoting Reform of University Entrance Examinations (Use of CBT in Individual Universities’ Admissions Selection, etc.) (FY 2022–FY 2024)
-
-
-
- May 18, 2023: “CBT Initiatives at The University of Electro-Communications,” FY 2023 National Conference of the Research Council on University Admissions (Link）
-
-
-
-
-
- August 27, 2023: “Practice of Global-Standard CBT Using IRT at The University of Electro-Communications,” Symposium at the Annual Conference of the Japan Association for Research on Testing (Link）
-
-
Questionnaire Results from the Trial Session for High School Students and CBT Sample Questions
-
-
-
- November 26, 2023: Questionnaire results for high school students after the CBT trial session (pdf）
-
-
-
-
-
- CBT sample questions and answers (pdf）
-
-
PCI Modules
The PCI developed as an outcome of the MEXT Commissioned Project for Promoting Reform of University Entrance Examinations, “Use of CBT in Individual Universities’ Admissions Selection, etc.,” is released under the MIT License (below).
Copyright (c) 2024 The University of Electro-Communications
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Installation Procedure
Set up a TAO 3.6.0 environment and install the following four PCI modules from the TAO administration screen.
Next, extract the ZIP file downloaded from the link below, and replace the file {TAO_ROOT_DIR}/tao/models/classes/export/implementation/CsvExporter.php in the TAO environment with the extracted CsvExporter.php file.
Project Results Report Symposium
The University of Electro-Communications: “A Trial of University Entrance Examinations Using Item-Bank-Based CBT Including the Subject ‘Informatics’”
On December 7, 2024, The University of Electro-Communications held a symposium on university entrance examinations using item-bank-based CBT (Computer Based Testing). CBT has attracted attention as an attempt to apply advances in educational technology to university entrance examinations, and 262 educators and researchers (57 in person and 205 online) participated either on site or online.
President Shunichi Tano delivered the opening remarks, and Professor Maomi Ueno reported on the results of implementing entrance examinations using the CBT system in the comprehensive selection and school-recommendation-based selection for the 2025 admissions cycle. During the Q&A session, participants, including teachers from Tokyo metropolitan high schools, asked many active questions. In addition, Yoshio Nitta, Academic Advisor at the Common Achievement Tests Organization for Medical Colleges, Yasuyo Sawaki, Professor at Waseda University, and Osamu Yoshimura, Professor at Nagasaki University, discussed the expectations and challenges of CBT in a panel discussion. In particular, there was lively discussion on the issue that current entrance examination rules are designed for paper-based tests, that many rules cannot be applied to CBT, and that these rules need to be organized and revised.
After a break, Professor Dai Nishigori of Saga University, Atsuki Osada, Assistant to the Administrative Director of Kanda University of International Studies, and Naohiro Terao, Associate Professor at the National Center for University Entrance Examinations, introduced the current CBT entrance examinations at partner universities under the CBT Utilization Collaboration Agreement. Specific examples were presented, including CBT entrance examinations using tablets at Saga University and online entrance examinations at Kanda University of International Studies. There was also an introduction to issues identified through interviews conducted by the National Center for University Entrance Examinations with universities implementing CBT. Finally, Vice President Satoshi Narumi delivered the closing remarks, bringing the symposium to a close.
This symposium was held with the support of the Ministry of Education, Culture, Sports, Science and Technology (MEXT) and with the cooperation of related educational institutions.
-
-
-
- Project Results Report Symposium – A Trial of University Entrance Examinations Using Item-Bank-Based CBT Including the Subject “Informatics” (Poster）
- Symposium handouts (pdf）
- Minutes of the External Evaluation Committee Panel Discussion (pdf）
- Answers to Questions Submitted Online (pdf）
- Symposium Questionnaire Results (pdf）
-
-
Collaboration on Use in University Admissions Selection
Please refer to the following page for activities and outputs related to collaboration on the use of CBT in university admissions selection.