Rapid technological, economic, and social transformations have fundamentally changed the competencies adults need to participate effectively in contemporary life (Díaz-García et al., 2022). The expansion of digital technologies has increased the volume, speed, and diversity of information that adults encounter, while digitalization has changed how information is accessed, communicated, and used in both work and everyday contexts. Adults are therefore increasingly expected to interpret multimodal information, evaluate quantitative evidence, navigate digital systems, and make decisions in situations characterized by changing information and multiple constraints (Yamashita et al., 2019). These changes are also reshaping the workplace, where mamy tasks increasingly require workers to process information, communicate effectively, and adapt changing condition, and solve non-routine problems alongside occupation-specific knowledge (Carnevale & Smith, 2013). Consequently, literacy, numeracy, and problem solving are becoming increasingly important not only for employment but also for adults’ capacity to make informed decisions in everyday context, including accessing and using information related to health and other aspects of daily life (Yamashita et al., 2019). Moreover, persistent low levels of these foundational competencies may contribute to cumulative social and employment disadvantages, particularly among young adults who experience prolonged disconnection from education and employment (van Vugt et al., 2024). These developments underscore the need for assessment approaches that capture how adults actually mobilize competencies in functional situations rather than merely measuring the retention of school-based knowledge.

The Programme for the International Assessment of Adult Competencies (PIAAC) was developed in response to this changing conception of adult competence and the growing need to assess how adults apply skills in real-life situations. PIAAC represents a large-scale international effort to examine how adults use information-processing skills across personal, social, and work-related situations (OECD, 2013; Schleicher, 2008). Its assessment framework focuses on literacy, numeracy, and problem solving in technology-rich environments, with subsequent developments increasingly emphasizing adaptive problem solving in dynamic and changing contexts (Kirsch & Lennon, 2017; OECD, 2024). This orientation is particularly relevant to contemporary adult competence because PIAAC does not treat these skills simply as bodies of knowledge to be recalled. The design of PIAAC is distinctive because assessment tasks are constructed around the interaction between cognitive processes, task characteristics, contexts, and information representations rather than around conventional disciplinary content alone (Kirsch et al., 2020). Research examining PIAAC through cognitive taxonomies has similarly demonstrated that its tasks encompass different levels and forms of cognitive engagement, highlighting the importance of understanding what respondents are required to do with information rather than simply whether an answer is correct (Boeren & Íñiguez-Berrozpe, 2022). Accordingly, PIAAC provides an important conceptual and methodological basis for assessing adult competencies as functional capacities that are mobilized in response to contextual demands, rather than as the mere retention of school-based knowledge.

A central implication of this approach is the importance of authenticity and context in assessment. Authentic assessment requires meaningful correspondence between assessment activities and the ways knowledge and skills are applied beyond formal testing situations. Critical elements include task realism, cognitive challenge, the use of knowledge in context, and opportunities to demonstrate performance through meaningful actions (Ashford-Rowe et al., 2014). This orientation is highly compatible with adult competency assessment because adults encounter literacy, numeracy, and problem-solving demands through activities such as interpreting public information, comparing prices, managing schedules, evaluating financial options, reading workplace documents, and navigating digital services. Studies using PIAAC data have shown that adult competencies are closely intertwined with educational, occupational, and social experiences (Kim, 2018; Lopes et al., 2020). From this perspective, the authenticity of an assessment context is not merely a decorative feature of an item; rather, it shapes the nature and complexity of the cognitive processes through which competence is demonstrated (Abbasi, 2025; Makeham & Lee, 2012; Villarroel et al., 2018). In authentic tasks, respondents must determine which information is relevant, select an appropriate strategy, and apply competencies within the constraints of a particular situation (Makeham & Lee, 2012; Oroh et al., 2025; Villarroel et al., 2018).

However, the use of international adult competency frameworks raises an important contextual challenge. Literacy practices are socially and culturally situated, and the meaning of competent performance cannot be fully separated from the contexts in which adults encounter and use information. Perry et al. (2020) have critically argued that large-scale literacy assessments may insufficiently represent the diversity and situated nature of everyday literacy practices. Differences in educational pathways and opportunities can also produce substantially different experiences of skill development among adult populations (Cosser, 2018), while cross-national and intergenerational variations demonstrate that literacy gaps are embedded within broader social structures (Kim, 2018). These concerns are particularly relevant to Indonesia, where improving adult competencies and aligning skills with changing labour-market demands remain significant policy priorities (OECD, 2021). Although PIAAC offers a robust international framework, assessment tasks developed for international comparison cannot simply be treated as ready-made learning or assessment resources for Indonesian adults. There is a need to translate the underlying competency constructs into authentic tasks that preserve the cognitive characteristics of PIAAC while drawing on information, services, occupations, and situations recognizable within Indonesian adult life.

The challenge becomes more complex when problem solving is situated in technology-rich environments. Digital problem solving involves more than technical proficiency; performance depends on how individuals search for information, sequence actions, evaluate alternatives, and modify strategies as a task unfolds. Vörös and Rouet (2016) demonstrated that strategy use is meaningfully related to performance in digital problem-solving tasks, while process-data research has revealed that sequences of actions can provide important evidence about how respondents approach PIAAC problems (Goldhammer et al., 2020; He & von Davier, 2015). Sequential patterns in problem-solving environments have also been associated with adults' background and employment characteristics (Liao et al., 2019). Similar concerns have emerged in professional contexts, where adults, including teachers, require opportunities to develop problem-solving competencies in technology-rich environments through meaningful professional and workplace learning experiences (De Wever et al., 2023). Consequently, a PIAAC-oriented assessment environment should not merely transfer paper-based questions to a computer screen. Rather, it should integrate authentic stimuli, multiple representations, flexible response formats, and digitally mediated situations that require users to access, interpret, apply, evaluate, and adapt information. In this study, a contextualized PIAAC assessment environment refers to an integrated system comprising contextualized PIAAC-oriented assessment items, a computer-based testing (CBT) system, an item book, and complementary dissemination media. The assessment items constitute the core assessment content; the CBT system provides the digital interface for presenting stimuli, recording responses, and delivering the assessment; the item book provides a supporting non-digital representation of the contextualized tasks; and dissemination media extend access to information about the assessment and adult competencies beyond the formal testing setting. Together, these components constitute the assessment environment developed and evaluated through the EDR process.

Despite extensive international research using PIAAC, much of the literature has concentrated on secondary analysis of large-scale datasets. Researchers have examined international competency patterns, sociodemographic differences, motivation-to-learn measurement, employment characteristics, and methodological procedures for analysing PIAAC data (Gorges et al., 2017; Jakubowski & Pokropek, 2019; Rammstedt et al., 2015). Other studies have advanced process-data analysis by investigating log files and action sequences generated during computer-based problem solving (Goldhammer et al., 2020; He & von Davier, 2015). These studies have substantially strengthened understanding of adult competencies and assessment behaviour. Nevertheless, comparatively limited attention has been given to the design process through which the PIAAC framework can be transformed into locally contextualized assessment items and an accessible digital assessment environment for adult learners, particularly in Indonesia. This constitutes a design-oriented research gap: existing evidence explains what PIAAC measures and how its data can be analysed, but provides less empirical guidance concerning how contextualized PIAAC-oriented assessment resources can be systematically designed, iteratively evaluated, and implemented in a specific national and cultural setting. Beyond the absence of contextualized assessment resources, there is also limited knowledge regarding the design principles required to translate an international competency framework into a coherent digital assessment environment for adult learners. Developing such an environment involves more than adapting item content; it requires aligning competency constructs, authentic contexts, cognitive processes, digital interfaces, response formats, and performance evidence into a coherent assessment system. Consequently, the challenge is not only whether contextualized PIAAC tasks can be produced, but also how they should be designed so that the intended competencies remain theoretically valid while becoming meaningful and accessible within local cultural settings.

Addressing this gap requires systematic development rather than direct translation or superficial modification of existing tasks. Robust literature synthesis is necessary to identify the theoretical and empirical principles underlying assessment design (García-Peñalvo, 2022; Turk, 2021), but theoretical alignment alone cannot ensure that an item is understandable, contextually meaningful, or technically functional for its intended users. The difficulty of an assessment task is also multidimensional and may be influenced by cognitive operations, the complexity of information, and the relationships respondents must construct when generating a response (Chen & Nunes, 2025). Therefore, locally contextualized PIAAC items need to be examined in terms of competency domain, cognitive process, context, stimulus, item format, expected response, and estimated difficulty. They also require iterative evaluation involving experts and prospective users. Such a process is particularly important for computer-based assessment because apparently minor issues—including ambiguous instructions, rigid answer validation, obscured visual information, or inconsistent response formats—may interfere with respondents' performance and confound the competencies intended to be measured. Educational Design Research (EDR) offers an appropriate methodological approach to address this challenge because it combines systematic product development with theory generation through iterative design, evaluation, and refinement (McKenney & Reeves, 2025; Plomp, 2013). Rather than treating assessment development as a linear production process, EDR enables researchers to investigate how theoretical principles are operationalized into practical design decisions while continuously incorporating evidence from experts and prospective users. Through successive cycles of formative evaluation, the resulting assessment environment is expected not only to satisfy criteria of validity and usability but also to generate empirically grounded and potentially transferable design principles for contextualizing international competency assessments.

Furthermore, adult competency assessment has potential educational functions beyond producing test scores. PIAAC research has generated important evidence concerning the relationship between adult skills, participation, employment, and sustainable development (OECD, 2016). Contextualized assessment tasks can make literacy, numeracy, and adaptive problem-solving demands more visible and recognizable to adults, educators, and other stakeholders. In this study, the contextualized tasks were also organized into an item book and complementary social media content to extend access to competency-related information beyond the formal assessment setting.

Accordingly, this study extends existing PIAAC research in three important ways. First, it shifts the focus from analysing assessment outcomes toward designing assessment environments grounded in authentic Indonesian contexts. Second, it demonstrates how literacy, numeracy, and adaptive problem-solving constructs can be systematically operationalized into contextualized digital assessment tasks through an iterative Educational Design Research process. Third, beyond producing a functional assessment platform, the study generates empirically informed design principles for contextualizing an international adult competency framework while preserving its underlying competency constructs. The contextualized items, computer-based testing system, item book, and dissemination materials therefore represent practical manifestations of the resulting design knowledge rather than the primary contribution of the study. These principles may inform future efforts to contextualize international competency frameworks in other adult education settings.

Against this background, the present study aimed to develop and evaluate a a contextualized PIAAC assessment environment for adults in Indonesia and, through this iterative Educational Design Research process, to generate empirically informed design principles for contextualizing an international adult competency assessment framework. Using Educational Design Research through the Plomp development phases and iterative formative evaluation, the study developed PIAAC-oriented items across literacy, numeracy, and problem-solving domains; integrated the items into a computer-based assessment environment; and refined the products through collaborative review, one-to-one evaluation, small-group evaluation, and field testing. The study also examined the dissemination of contextualized PIAAC tasks through a structured item book and social media content. Accordingly, this study addresses the following research question: How can a contextualized PIAAC assessment environment be systematically designed and iteratively evaluated to produce valid, practical, and usablility literacy, numeracy, and problem-solving tasks for adults in Indonesia?

This study employed Educational Design Research (EDR) to develop a web-based learning environment that supports the development of adults' competencies assessed in the Programme for the International Assessment of Adult Competencies (PIAAC). Educational Design Research is an intervention-oriented research approach that aims to simultaneously address authentic educational problems while generating practical solutions and transferable design knowledge through iterative cycles of analysis, design, evaluation, and refinement (McKenney & Reeves, 2025). Unlike conventional product development studies, EDR emphasizes continuous interaction between theoretical understanding and practical implementation, allowing the developed intervention to evolve based on empirical evidence collected throughout the research process (McKenney & Reeves, 2025).

The development process followed the Plomp development model, which consists of three interconnected phases: Preliminary Research, Prototyping Phase, and Assessment Phase. The preliminary research phase focused on identifying educational needs, analysing the PIAAC framework, and establishing design principles for the learning environment. The prototyping phase involved iterative development and revision of the prototype through Tessmer's formative evaluation, including self-evaluation, expert review, one-to-one evaluation, and small-group evaluation. Finally, the assessment phase evaluated the quality of the final product in terms of validity, practicality, and initial feasibility through field implementation. The integration of Plomp and Tessmer enables systematic product refinement while ensuring that the resulting learning environment is theoretically grounded and empirically validated.

The primary product developed in this study was a web-based PIAAC learning environment that integrates locally contextualized PIAAC assessment items, a digital learning platform, printed learning resources, and social media learning content to facilitate adult learning.

This study successfully produced contextualized PIAAC-type assessment resources that were valid and practical, and that demonstrated potential effects on adults’ literacy, numeracy, and adaptive problem-solving competencies, particularly among lecturers, prospective teachers, and teachers in Palembang. The development process began with an initial pool of 100 items, from which 93 items were retained as the final contextualized assessment items, comprising 32 literacy, 31 numeracy, and 30 problem-solving items. The main outputs comprised a book containing the 93 final PIAAC-type items, a web-based assessment system accessible through piaac.id and Google Forms, social media dissemination through Instagram, and direct dissemination activities through courses and workshops. These products provide an integrated environment for assessing, introducing, and strengthening adult competencies through authentic personal, professional, and societal contexts.

Further development should focus on transforming the web-based assessment system from a testing platform into an online training environment that explains the purpose, relevance, and application of PIAAC competencies in everyday and workplace situations. Dissemination should also be expanded through the CBT and additional social media platforms, including X, to reach a wider population of learners and workers aged 16–65. In addition, PIAAC-oriented content should be integrated as a regular component of the PMRI–PISA Summer Course and relevant university courses to support the sustainability and broader educational use of the developed resources.