ISO/TS 37444:2023
(Main)Electronic fee collection — Charging performance framework
Electronic fee collection — Charging performance framework
This document defines the charging performance metrics to be used during the evaluation or on-going monitoring of an electronic fee collection (EFC) system and the examination framework for the measurement of these metrics. It specifies a method for the specification and documentation of a specific examination framework which can be used by the responsible entity to evaluate charging performance for a particular information exchange interface or for overall charging performance within a toll scheme. The following scheme types are within the scope of this document: a) discrete schemes; b) continuous schemes (autonomous type of systems). This document defines measurements only on standardized interfaces. This document defines metrics for the charging performance of EFC systems in terms of the level of errors associated with charging computation. This document describes a set of metrics with definitions, principles and formulations, which together make up a reference framework for the establishment of requirements for EFC systems and the subsequent examination of charging performance. This document defines metrics for the following information exchanges: — charge reports (including usage evidence); — toll declarations; — exception lists; — billing details and associated event data; — payment claims on the level of service user accounts; — end-to-end metrics which assess the overall performance of the charging process. These metrics focus solely on the outcome of the charging process, i.e. the amount charged in relation to a pre-measured or theoretically correct amount, rather than intermediate variables from various components as sensors, such as positioning accuracy, signal range or optical resolution. This approach ensures comparable results for each metric in all relevant situations. The following aspects are outside the scope of this document. — Definition of specific numeric performance bounds, or average or worst-case error bounds in percentage or monetary units. — Specification of a common reference system which would be required for comparison of performance between systems. — Measurements on proprietary interfaces. NOTE It is not possible to define standardized metrics on such system properties. Neither is it possible to define metrics for parts of the charging processing chain which are considered to be the internal matter of an interoperability partner, such as: — equipment performance, e.g. for on-board equipment (OBE), roadside equipment (RSE) or data centres such as signal range, optical resolution or computing system availability; — position performance metrics: the quality of data generated by position sensors is considered as an internal aspect of the GNSS front end. It is masked by correction algorithms, filtering, inferring of data and the robustness of the charge object recognition algorithms. — The evaluation of the expected performance of a system based on modelling and measured data from a trial at another place.
Perception de télépéage — Cadre de performance d'imputation
General Information
Relations
Buy Standard
Standards Content (Sample)
TECHNICAL ISO/TS
SPECIFICATION 37444
First edition
2023-06
Electronic fee collection — Charging
performance framework
Perception de télépéage — Cadre de performance d'imputation
Reference number
ISO/TS 37444:2023(E)
© ISO 2023
---------------------- Page: 1 ----------------------
ISO/TS 37444:2023(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO 2023
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii
© ISO 2023 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/TS 37444:2023(E)
Contents Page
Foreword .v
Introduction . vi
1 Scope . 1
2 Normative references . 2
3 Terms and definitions . 2
4 Symbols and abbreviated terms.6
5 Examination framework .7
5.1 General . 7
5.2 Method for defining a specific examination framework . 7
5.2.1 General . 7
5.2.2 Selection of metrics to be evaluated . 8
5.2.3 Definition of environmental conditions and performance requirements . 9
5.2.4 Determination of required sample sizes . 9
5.2.5 Selection of methods for generating charging input and reference data . 9
5.2.6 Determination of test routes and trips . 9
5.2.7 Definition of measurement time period . 10
5.2.8 Documentation of the specific examination framework . 10
5.3 Sources of data. 10
5.4 Methods of generating charging input . 13
5.4.1 General .13
5.4.2 Predefined routes (identifier: “PVP”) . 14
5.4.3 Reference system (used in combination with identifiers: “PVR” and “UVR”) .15
5.4.4 Simulated OBE/FE (identifier: “SO”) . 16
5.4.5 Dedicated OBE testing (identifier: “DO”). 16
6 Charging performance metrics .17
6.1 General . 17
6.2 Metric identification .20
6.3 Charge report metrics . 21
6.3.1 General . 21
6.3.2 Metrics relevant for all schemes . 21
6.3.3 Metrics only applicable to discrete schemes .22
6.3.4 Metrics applicable to continuous schemes . 23
6.4 Toll declaration metrics . 24
6.4.1 General . 24
6.4.2 Metrics relevant for all schemes . 24
6.4.3 Metrics only applicable to discrete schemes . 25
6.4.4 Metrics applicable to continuous schemes . 26
6.5 Billing details metrics . 27
6.6 Payment claim metrics . 29
6.7 Exception list metrics .30
6.8 User account metrics . 31
6.9 End-to-end metrics . 32
6.10 Applicability of metrics scheme types . 32
6.11 Charging metric selection tables . 37
6.11.1 General . 37
6.11.2 Discrete . 37
6.11.3 Autonomous discrete .39
6.11.4 Autonomous continuous . 41
7 Examination tests .43
7.1 General . 43
7.2 Technology-independent tests. 43
7.2.1 General . 43
iii
© ISO 2023 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/TS 37444:2023(E)
7.2.2 ET-CM-E2E-1 E2E — Correct charging rate . 43
7.2.3 ET-CM-E2E-2 E2E — Overcharging rate .44
7.2.4 ET-CM-E2E-3 E2E — Undercharging rate . 45
7.2.5 ET-CM-E2E-4 E2E — Late charging rate .46
7.2.6 ET-CM-UA-1 UA — Correct charging rate .46
7.2.7 ET-CM-UA-2 UA — Overcharging rate . 47
7.2.8 ET-CM-UA-3 UA — Undercharging rate .48
7.2.9 ET-CM-UA-4 UA — Accurate application of payments and refunds .49
7.2.10 ET-CM-UA-5 UA — Accurate personalization of OBEs .49
7.2.11 ET-CM-EL-1 EL — Correct exception list generation rate .50
7.2.12 ET-CM-EL-2 EL — Incorrect exception list generation rate .50
7.2.13 ET-CM-PC-1 PC — Correct charging rate . 51
7.2.14 ET-CM-PC-2 PC — Overcharging Rate . 52
7.2.15 ET-CM-PC-3 PC — Undercharging rate . 53
7.2.16 ET-CM-PC-4 PC — Latency — TC . 53
7.2.17 ET-CM-PC-5 PC — Late payment claims rate .54
7.2.18 ET-CM-PC-6 PC — Rejected payment claim rate . 55
7.2.19 ET-CM-BD-1 BD — Correct charging rate . 55
7.2.20 ET-CM-BD-2 BD — Overcharging rate.56
7.2.21 ET-CM-BD-3 BD — Undercharging rate . 57
7.2.22 ET-CM-BD-4 BD — Incorrect charging rate . 57
7.2.23 ET-CM-BD-5 BD — Latency — TC .58
7.2.24 ET-CM-BD-6 BD — Late billing details rate . 59
7.2.25 ET-CM-BD-7 BD — Rejected billing details rate . 59
7.2.26 ET-CM-BD-8 BD — Incorrectly rejected billing details rate.60
7.2.27 ET-CM-BD-9 BD — Inferred billing details rate .60
7.2.28 ET-CM-CR-1 CR— Usage evidence availability . 61
7.2.29 ET-CM-CR-2 CR— Usage evidence integrity . 61
7.2.30 ET-CM-CR-3 CR- Usage evidence time-to-first-fix . 62
7.3 Technology-dependent tests .63
7.3.1 Autonomous discrete specific examination tests .63
7.3.2 Discrete — Optional toll declaration metrics .73
7.3.3 Autonomous continuous specific examination tests . 76
Annex A (informative) Examination test documentation template .86
Annex B (informative) Examination framework considerations .87
Annex C (informative) Statistical considerations .90
Annex D (informative) Methods for reducing sample sizes during the evaluation phase .95
Annex E (informative) Examples of specific examination frameworks .99
Annex F (informative) Defining performance requirements .110
Bibliography . 118
iv
© ISO 2023 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/TS 37444:2023(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see
www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 204, Intelligent transport systems, in
collaboration with the European Committee for Standardization (CEN) Technical Committee CEN/TC
278, Intelligent transport systems, in accordance with the Agreement on technical cooperation between
ISO and CEN (Vienna Agreement).
This first edition cancels and replaces the second editions ISO/TS 17444-1:2017 and ISO/TS 17444-2:2017,
which have been technically revised.
The main changes are as follows:
— the resulting document has been renumbered as ISO/TS 37444;
— various editorial changes have been made to improve the readability of the text;
— a technology-neutral definition of metrics and examination tests has been applied, which also includes
support for tolling systems based on automatic number plate recognition (ANPR) technology;
— terminology and references to other documents have been updated.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.
v
© ISO 2023 – All rights reserved
---------------------- Page: 5 ----------------------
ISO/TS 37444:2023(E)
Introduction
0.1 General
Electronic tolling systems are complex distributed systems involving critical technology such as
dedicated short-range communication (DSRC), camera-based technology (e.g. automatic number plate
recognition, ANPR) and global navigation satellite systems (GNSS). These technologies are all subject
to a certain behaviour that can affect the computation of the charges. Thus, to protect the interests
of the different stakeholders involved, particularly toll service users (SUs) and toll chargers (TCs), it
is essential to define metrics that measure the performance of the system in terms of computation
of charges, and that ensure that the potential resulting errors are acceptable. These metrics will be
a useful tool for establishing requirements for the systems and for examining the system capabilities
during acceptance and throughout the operational life of the system.
In addition, to ensure the interoperability of different systems, it is necessary to agree on common
metrics for use and on the actual values that define the required acceptable performances. However,
these points are not covered in this document.
Instead, this document is defined as a toolbox standard of examination tests, with a method for defining
and documenting a specific examination framework to meet specific needs. The detailed choice of the set
of examination tests within an examination framework depends on the application and the respective
context. Conformance to this document means using the definitions and prescriptions laid out in this
document whenever the respective system aspects are subjected to performance measurements, rather
than using other definitions and examination methods.
0.2 Charging performance metrics
This document also defines a set of charging performance metrics with definitions, principles and
formulae, which together make up a reference framework for the establishment of requirements for
electronic fee collection (EFC) systems and their later examination of the charging performance.
These charging performance metrics are intended for use with any toll scheme, regardless of its
technical underpinnings, system architecture, tariff structure, geographical coverage or organizational
model. They are defined to treat technical details that can differ between technologies as a “black
box”. They focus solely on the outcome of the charging process, i.e. the amount charged in relation
to a pre-measured or theoretically correct amount, rather than intermediate variables from various
components as sensors, such as positioning accuracy, signal range or optical resolution. This approach
ensures comparable results for each metric in all relevant situations.
The metrics are designed to cover the information exchanged on the front-end (FE) interface and the
interoperability interfaces between toll service providers (TSPs) and TCs, as well as information on the
end-to-end level.
Metrics for the following information exchanges are defined:
— charge reports (including usage evidence);
— toll declarations;
— billing details and associated event data;
— payment claims on the level of user accounts;
— exception lists;
— end-to-end metrics which assess the overall performance of the charging process.
The proposed metrics are specifically addressed to protect the interests of the actors in a toll system,
such as TSPs, TCs and SUs. They can be used to define requirements (e.g. for requests for proposals) and
for performance assessments.
vi
© ISO 2023 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/TS 37444:2023(E)
Toll schemes take on various forms as identified in ISO 17573-1 and ISO 12855. In order to create a
uniform performance metric specification, toll schemes are grouped into two classes based on the
character of their primary charging variable:
— charging based on discrete events (charges associated to the fact that a vehicle is crossing or
standing within a certain zone);
— charging based on a continuous measurement (duration or distance).
The following are examples of discrete (event-based) toll schemes.
— Single object charging: a road section, bypass, bridge, tunnel, mountain pass or even a ferry, charged
per passage.
EXAMPLE 1 Most tolled bridges belong to this category.
— Closed road charging: a fixed amount is charged for a certain combination of entry and exit on a
motorway or other closed road network.
EXAMPLE 2 Many of the motorways in Southern Europe belong to this category.
— Discrete road links charging: determined by use of specified road links, whether or not they are
used in their entirety.
EXAMPLE 3 Heavy goods vehicle (HGV) charge in Germany.
— Charging for cordon crossing: triggered by passing in or out through a cordon that encircles a city
core, for example.
EXAMPLE 4 Congestion and infrastructure charging schemes in Stockholm and Gothenburg (Sweden).
The following are examples of continuous toll schemes.
— Charging based on direct distance measurement: defined as an amount per km driven.
EXAMPLE 5 HGV charge in Switzerland and US basic vehicle miles-travelled toll systems concepts.
— Charging based on direct distance measurement in different tariff zones or road types: defined
as an amount per km driven, with different tariffs applying in different zones or on different road
types. This is a widely discussed approach, also known as time-distance-place charging, and is
under consideration in European countries.
EXAMPLE 6 OReGO, the pilot programme in Oregon, North America.
— Time in use charge: determined by the accumulated time a vehicle has been in operation, or
alternatively, by the time the vehicle has been present inside a predefined zone.
In all of these toll schemes, tolls can additionally vary as a function of vehicle class characteristics (such
as trailer presence, number of axles, taxation class and operating function) and depending on time of
day or day of week, such that, for example, tariffs are higher in rush hour and lower on weekends.
With this degree of complexity, it is not surprising to find that the attempts to evaluate and compare
technical solutions for SU charging have been made on an individual basis each time a procurement
or study is initiated, and with only limited ability to reuse prior comparisons made by other testing
entities.
The identification of different types of schemes as proposed in the ISO 17575 series and their grouping
in the mentioned two classes is described in Table 1. Table 1 also identifies the examples mentioned
above.
vii
© ISO 2023 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/TS 37444:2023(E)
Table 1 — Toll scheme designs grouped according to scheme categories
Examples Scheme type ISO 17575 series category
Single object charging Discrete Sectioned roads pricing
Closed road charging Discrete Sectioned roads pricing
Discrete road links charging Discrete Sectioned roads pricing
Charging for cordon crossing Discrete Cordon pricing
Time in use charge Continuous Area pricing — time
Cumulative distance charge Continuous Area pricing — distance
Charging for cumulative distance in different zones (or Continuous Area pricing — distance
by road type)
0.3 Examination framework
The examination framework that is defined in this document is designed for measuring the metrics
defined in Clause 6. The general aim is to achieve a maximum comparability and reproducibility
of the results without restricting the technological choices in system design. Specific examination
frameworks can be defined for the evaluation and monitoring phases of a project due to the differences
in the availability of equipped vehicles.
a) Evaluation phase
The evaluation phase encompasses system evaluation and selection, as well as commissioning and
ramp-up during implementation. Important aspects of this phase are:
— relatively small sample sizes; and
— well-controlled behaviour of test vehicles.
b) Monitoring phase
After the system has gone into operation, its behaviour needs to be monitored for several reasons,
such as fine-tuning of the system performance, monitoring of service level agreements (SLAs) between
contractual partners (supplier, TC, TSP, etc.). In this phase, the following system aspects can be
expected:
— very large sample sizes possible, but with unknown behaviour of the vehicles;
— in principle all measurements from implementation phase possible, too.
0.4 Reader's guide
To understand the content of this document, the reader should be aware of the methodology and
assumptions used to develop the examination framework; therefore, a suggested reading order is given
below.
a) Annex B provides details of the underlying considerations for developing the examination
framework.
b) Annex C provides background statistical information which will enable the reader to determine
sample sizes and confidence limits based on the defined performance requirements.
c) Clause 5 provides the definition of the examination framework for the evaluation of charging
performance.
d) Clause 6 provides definitions of charging metrics and their applicability to the scheme types
described above.
e) Clause 7 contains the toolbox of examination tests for the evaluation of charging performance for
the identified scheme types.
viii
© ISO 2023 – All rights reserved
---------------------- Page: 8 ----------------------
ISO/TS 37444:2023(E)
f) Annex A contains an example template for the documentation of examination tests and their
results.
g) Annex D contains methods which can be used to reduce the required sample sizes for metrics with
high and low probabilities during the evaluation phase.
h) Annex E provides examples of specific examination frameworks which have been developed in
accordance with the methodology in 5.2.
ix
© ISO 2023 – All rights reserved
---------------------- Page: 9 ----------------------
TECHNICAL SPECIFICATION ISO/TS 37444:2023(E)
Electronic fee collection — Charging performance
framework
1 Scope
This document defines the charging performance metrics to be used during the evaluation or on-
going monitoring of an electronic fee collection (EFC) system and the examination framework for the
measurement of these metrics.
It specifies a method for the specification and documentation of a specific examination framework
which can be used by the responsible entity to evaluate charging performance for a particular
information exchange interface or f
...
TECHNICAL SPECIFICATION
ISO/TC 204
ISO/DTS 37444:2022 (E)
Date: 2022-12
Secretariat: ANSI
Electronic fee collection –— Charging performance framework
Perception de télépéage — Cadre de performance d'imputation
---------------------- Page: 1 ----------------------
ISO/DTS 37444:2022 (E)
© ISO 2022
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this
publication may be reproduced or utilized otherwise in any form or by any means, electronic or mechanical,
including photocopying, or posting on the internet or an intranet, without prior written permission. Permission
can be requested from either ISO at the address below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: + 41 22 749 01 11
Fax: + 41 22 749 09 47
E-mail: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO 2022 – All rights reserved
ii © ISO 2022 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/DTS 37444:2022(E)
Contents
Foreword . vi
Introduction . vii
1 Scope . 1
2 Normative references . 2
3 Terms and definitions . 2
4 Symbols and abbreviated terms . 6
5 Examination framework . 8
5.1 General . 8
5.2 Method for defining a specific examination framework . 8
5.2.1 General . 8
5.2.2 Selection of metrics to be evaluated . 9
5.2.3 Definition of environmental conditions and performance requirements . 10
5.2.4 Determination of required sample sizes . 10
5.2.5 Selection of methods for generating charging input and reference data. 10
5.2.6 Determination of test routes and trips . 11
5.2.7 Definition of measurement time period . 11
5.2.8 Documentation of the specific examination framework . 11
5.3 Sources of data . 11
5.4 Methods of generating charging input . 16
5.4.1 General . 16
5.4.2 Predefined routes (identifier: “PVP”) . 17
5.4.3 Reference system (used in combination with identifiers: “PVR” and “UVR”) . 17
5.4.4 Simulated OBE/FE (identifier: “SO”) . 19
5.4.5 Dedicated OBE testing (identifier: “DO”) . 19
6 Charging performance metrics . 20
6.1 General . 20
6.2 Metric identification . 24
6.3 Charge report metrics . 25
6.3.1 General . 25
6.3.2 Metrics relevant for all schemes . 25
6.3.3 Metrics only applicable to discrete schemes . 26
6.3.4 Metrics applicable to continuous schemes . 27
6.4 Toll declaration metrics . 28
6.4.1 General . 28
© ISO 2022 – All rights reserved iii
© ISO 2022 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/DTS 37444:2022 (E)
6.4.2 Metrics relevant for all schemes . 29
6.4.3 Metrics only applicable to discrete schemes . 30
6.4.4 Metrics applicable to continuous schemes . 31
6.5 Billing details metrics . 32
6.6 Payment claim metrics . 34
6.7 Exception list metrics . 36
6.8 User account metrics . 37
6.9 End-to-End metrics . 38
6.10 Applicability of metrics scheme types . 39
6.11 Charging metric selection tables . 43
6.11.1 General . 43
6.11.2 Discrete . 43
6.11.3 Autonomous discrete . 45
6.11.4 Autonomous continuous . 48
7 Examination tests. 51
7.1 Technology-independent tests . 51
7.1.1 General . 51
7.1.2 ET-CM-E2E-1 E2E — Correct charging rate . 52
7.1.3 ET-CM-E2E-2 E2E — Overcharging rate . 53
7.1.4 ET-CM-E2E-3 E2E — Undercharging rate . 53
7.1.5 ET-CM-E2E-4 E2E — Late charging rate . 54
7.1.6 ET-CM-UA-1 UA — Correct charging rate . 55
7.1.7 ET-CM-UA-2 UA — Overcharging rate . 56
7.1.8 ET-CM-UA-3 UA — Undercharging rate . 57
7.1.9 ET-CM-UA-4 UA — Accurate application of payments and refunds . 57
7.1.10 ET-CM-UA-5 UA — Accurate personalization of OBEs . 58
7.1.11 ET-CM-EL-1 EL — Correct exception list generation rate . 59
7.1.12 ET-CM-EL-2 EL — Incorrect exception list generation rate. 59
7.1.13 ET-CM-PC-1 PC — Correct charging rate . 60
7.1.14 ET-CM-PC-2 PC — Overcharging Rate . 61
7.1.15 ET-CM-PC-3 PC — Undercharging Rate . 62
7.1.16 ET-CM-PC-4 PC — Latency — TC . 62
7.1.17 ET-CM-PC-5 PC — Late payment claims rate . 63
7.1.18 ET-CM-PC-6 PC — Rejected payment claim rate . 64
7.1.19 ET-CM-BD-1 BD — Correct charging rate . 64
iv © ISO 2022 – All rights reserved
iv © ISO 2022 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/DTS 37444:2022(E)
7.1.20 ET-CM-BD-2 BD — Overcharging rate . 65
7.1.21 ET-CM-BD-3 BD — Undercharging rate . 66
7.1.22 ET-CM-BD-4 BD — Incorrect charging rate . 67
7.1.23 ET-CM-BD-5 BD — Latency — TC . 67
7.1.24 ET-CM-BD-6 BD — Late billing details rate . 68
7.1.25 ET-CM-BD-7 BD — Rejected billing details rate . 68
7.1.26 ET-CM-BD-8 BD — Incorrectly rejected billing details rate . 69
7.1.27 ET-CM-BD-9 BD — Inferred billing details rate . 69
7.1.28 ET-CM-CR-1 CR— Usage evidence availability . 70
7.1.29 ET-CM-CR-2 CR— Usage evidence integrity . 70
7.1.30 ET-CM-CR-3 CR- Usage evidence time-to-first-fix . 71
7.2 Technology-dependent tests . 72
7.2.1 Autonomous discrete specific examination tests . 72
7.2.2 Discrete — Optional toll declaration metrics . 82
7.2.3 Autonomous continuous specific examination tests . 86
Annex A (informative) Examination test documentation template . 96
Annex B (informative) Examination framework considerations . 97
Annex C (informative) Statistical considerations . 100
Annex D (informative) Methods for reducing sample sizes during the evaluation phase . 105
Annex E (informative) Examples of specific examination frameworks . 109
Annex F (informative) Defining performance requirements . 122
Bibliography . 131
Foreword . ix
Introduction . xi
1 Scope . 1
2 Normative references . 2
3 Terms and definitions . 2
4 Symbols and abbreviated terms . 6
5 Examination framework . 8
5.1 General . 8
5.2 Method for defining a specific examination framework . 8
5.2.1 General . 8
5.2.2 Selection of metrics to be evaluated . 10
5.2.3 Definition of environmental conditions and performance requirements . 11
© ISO 2022 – All rights reserved v
© ISO 2022 – All rights reserved v
---------------------- Page: 5 ----------------------
ISO/DTS 37444:2022 (E)
5.2.4 Determination of required sample sizes . 11
5.2.5 Selection of methods for generating charging input and reference data. 11
5.2.6 Determination of test routes and trips . 12
5.2.7 Definition of measurement time period . 12
5.2.8 Documentation of the specific examination framework . 12
5.3 Sources of data . 12
5.4 Methods of generating charging input . 18
5.4.1 General . 18
5.4.2 Predefined routes (identifier: “PVP”) . 19
5.4.3 Reference system (used in combination with identifiers: “PVR” and “UVR”) . 20
5.4.4 Simulated OBE/FE (identifier: “SO”) . 21
5.4.5 Dedicated OBE testing (identifier: “DO”) . 22
6 Charging performance metrics . 22
6.1 General . 22
6.2 Metric identification . 29
6.3 Charge report metrics . 30
6.3.1 General . 30
6.3.2 Metrics relevant for all schemes . 30
6.3.3 Metrics only applicable to discrete schemes . 31
6.3.4 Metrics applicable to continuous schemes . 32
6.4 Toll declaration metrics . 33
6.4.1 General . 33
6.4.2 Metrics relevant for all schemes . 34
6.4.3 Metrics only applicable to discrete schemes . 35
6.4.4 Metrics applicable to continuous schemes . 36
6.5 Billing details metrics . 37
6.6 Payment claim metrics . 40
6.7 Exception list metrics . 41
6.8 User account metrics . 42
6.9 End-to-end metrics . 43
6.10 Applicability of metrics scheme types . 44
6.11 Charging metric selection tables . 49
6.11.1 General . 49
6.11.2 Discrete . 49
6.11.3 Autonomous discrete . 51
vi © ISO 2022 – All rights reserved
vi © ISO 2022 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/DTS 37444:2022(E)
6.11.4 Autonomous continuous . 54
7 Examination tests. 57
7.1 General . 57
7.2 Technology-independent tests . 57
7.2.1 General . 57
7.2.2 ET-CM-E2E-1 E2E — Correct charging rate . 58
7.2.3 ET-CM-E2E-2 E2E — Overcharging rate . 59
7.2.4 ET-CM-E2E-3 E2E — Undercharging rate . 59
7.2.5 ET-CM-E2E-4 E2E — Late charging rate . 60
7.2.6 ET-CM-UA-1 UA — Correct charging rate . 61
7.2.7 ET-CM-UA-2 UA — Overcharging rate . 62
7.2.8 ET-CM-UA-3 UA — Undercharging rate . 63
7.2.9 ET-CM-UA-4 UA — Accurate application of payments and refunds . 64
7.2.10 ET-CM-UA-5 UA — Accurate personalization of OBEs . 64
7.2.11 ET-CM-EL-1 EL — Correct exception list generation rate . 65
7.2.12 ET-CM-EL-2 EL — Incorrect exception list generation rate. 66
7.2.13 ET-CM-PC-1 PC — Correct charging rate . 67
7.2.14 ET-CM-PC-2 PC — Overcharging Rate . 68
7.2.15 ET-CM-PC-3 PC — Undercharging rate . 69
7.2.16 ET-CM-PC-4 PC — Latency — TC . 70
7.2.17 ET-CM-PC-5 PC — Late payment claims rate . 71
7.2.18 ET-CM-PC-6 PC — Rejected payment claim rate . 71
7.2.19 ET-CM-BD-1 BD — Correct charging rate . 72
7.2.20 ET-CM-BD-2 BD — Overcharging rate . 73
7.2.21 ET-CM-BD-3 BD — Undercharging rate . 74
7.2.22 ET-CM-BD-4 BD — Incorrect charging rate . 75
7.2.23 ET-CM-BD-5 BD — Latency — TC . 75
7.2.24 ET-CM-BD-6 BD — Late billing details rate . 76
7.2.25 ET-CM-BD-7 BD — Rejected billing details rate . 76
7.2.26 ET-CM-BD-8 BD — Incorrectly rejected billing details rate . 77
7.2.27 ET-CM-BD-9 BD — Inferred billing details rate . 78
7.2.28 ET-CM-CR-1 CR— Usage evidence availability . 78
7.2.29 ET-CM-CR-2 CR— Usage evidence integrity . 79
7.2.30 ET-CM-CR-3 CR- Usage evidence time-to-first-fix . 80
7.3 Technology-dependent tests . 81
© ISO 2022 – All rights reserved vii
© ISO 2022 – All rights reserved vii
---------------------- Page: 7 ----------------------
ISO/DTS 37444:2022 (E)
7.3.1 Autonomous discrete specific examination tests . 81
7.3.2 Discrete — Optional toll declaration metrics . 91
7.3.3 Autonomous continuous specific examination tests . 94
Annex A (informative) Examination test documentation template . 105
Annex B (informative) Examination framework considerations . 106
Annex C (informative) Statistical considerations .
...
FINAL
TECHNICAL ISO/DTS
DRAFT
SPECIFICATION 37444
ISO/TC 204
Electronic fee collection — Charging
Secretariat: ANSI
performance framework
Voting begins on:
2022-12-23
Perception de télépéage — Cadre de performance d'imputation
Voting terminates on:
2023-03-17
RECIPIENTS OF THIS DRAFT ARE INVITED TO
SUBMIT, WITH THEIR COMMENTS, NOTIFICATION
OF ANY RELEVANT PATENT RIGHTS OF WHICH
THEY ARE AWARE AND TO PROVIDE SUPPOR TING
DOCUMENTATION.
IN ADDITION TO THEIR EVALUATION AS
Reference number
BEING ACCEPTABLE FOR INDUSTRIAL, TECHNO-
ISO/DTS 37444:2022(E)
LOGICAL, COMMERCIAL AND USER PURPOSES,
DRAFT INTERNATIONAL STANDARDS MAY ON
OCCASION HAVE TO BE CONSIDERED IN THE
LIGHT OF THEIR POTENTIAL TO BECOME STAN-
DARDS TO WHICH REFERENCE MAY BE MADE IN
NATIONAL REGULATIONS. © ISO 2022
---------------------- Page: 1 ----------------------
ISO/DTS 37444:2022(E)
FINAL
TECHNICAL ISO/DTS
DRAFT
SPECIFICATION 37444
ISO/TC 204
Electronic fee collection — Charging
Secretariat: ANSI
performance framework
Voting begins on:
Perception de télépéage — Cadre de performance d'imputation
Voting terminates on:
COPYRIGHT PROTECTED DOCUMENT
© ISO 2022
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on
the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address below
or ISO’s member body in the country of the requester.
RECIPIENTS OF THIS DRAFT ARE INVITED TO
ISO copyright office
SUBMIT, WITH THEIR COMMENTS, NOTIFICATION
OF ANY RELEVANT PATENT RIGHTS OF WHICH
CP 401 • Ch. de Blandonnet 8
THEY ARE AWARE AND TO PROVIDE SUPPOR TING
CH-1214 Vernier, Geneva
DOCUMENTATION.
Phone: +41 22 749 01 11
IN ADDITION TO THEIR EVALUATION AS
Reference number
Email: copyright@iso.org
BEING ACCEPTABLE FOR INDUSTRIAL, TECHNO
ISO/DTS 37444:2022(E)
Website: www.iso.org
LOGICAL, COMMERCIAL AND USER PURPOSES,
DRAFT INTERNATIONAL STANDARDS MAY ON
Published in Switzerland
OCCASION HAVE TO BE CONSIDERED IN THE
LIGHT OF THEIR POTENTIAL TO BECOME STAN
DARDS TO WHICH REFERENCE MAY BE MADE IN
ii
© ISO 2022 – All rights reserved
NATIONAL REGULATIONS. © ISO 2022
---------------------- Page: 2 ----------------------
ISO/DTS 37444:2022(E)
Contents Page
Foreword .v
Introduction . vi
1 Scope . 1
2 Normative references . 2
3 Terms and definitions . 2
4 Symbols and abbreviated terms.6
5 Examination framework .7
5.1 General . 7
5.2 Method for defining a specific examination framework . 7
5.2.1 General . 7
5.2.2 Selection of metrics to be evaluated . 8
5.2.3 Definition of environmental conditions and performance requirements . 9
5.2.4 Determination of required sample sizes . 9
5.2.5 Selection of methods for generating charging input and reference data . 9
5.2.6 Determination of test routes and trips . 9
5.2.7 Definition of measurement time period . 10
5.2.8 Documentation of the specific examination framework . 10
5.3 Sources of data. 10
5.4 Methods of generating charging input . 13
5.4.1 General .13
5.4.2 Predefined routes (identifier: “PVP”) . 14
5.4.3 Reference system (used in combination with identifiers: “PVR” and “UVR”) .15
5.4.4 Simulated OBE/FE (identifier: “SO”) . 16
5.4.5 Dedicated OBE testing (identifier: “DO”). 16
6 Charging performance metrics .17
6.1 General . 17
6.2 Metric identification .20
6.3 Charge report metrics . 21
6.3.1 General . 21
6.3.2 Metrics relevant for all schemes . 21
6.3.3 Metrics only applicable to discrete schemes .22
6.3.4 Metrics applicable to continuous schemes . 23
6.4 Toll declaration metrics . 24
6.4.1 General . 24
6.4.2 Metrics relevant for all schemes . 24
6.4.3 Metrics only applicable to discrete schemes . 25
6.4.4 Metrics applicable to continuous schemes . 26
6.5 Billing details metrics . 27
6.6 Payment claim metrics . 29
6.7 Exception list metrics .30
6.8 User account metrics . 31
6.9 Endtoend metrics . 32
6.10 Applicability of metrics scheme types . 32
6.11 Charging metric selection tables . 37
6.11.1 General . 37
6.11.2 Discrete . 37
6.11.3 Autonomous discrete .39
6.11.4 Autonomous continuous . 41
7 Examination tests .43
7.1 General . 43
7.2 Technology-independent tests. 43
7.2.1 General . 43
iii
© ISO 2022 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/DTS 37444:2022(E)
7.2.2 ETCME2E1 E2E — Correct charging rate . 43
7.2.3 ETCME2E2 E2E — Overcharging rate .44
7.2.4 ETCME2E3 E2E — Undercharging rate . 45
7.2.5 ETCME2E4 E2E — Late charging rate .46
7.2.6 ETCMUA1 UA — Correct charging rate .46
7.2.7 ETCMUA2 UA — Overcharging rate . 47
7.2.8 ETCMUA3 UA — Undercharging rate .48
7.2.9 ET-CM-UA-4 UA — Accurate application of payments and refunds .49
7.2.10 ETCMUA5 UA — Accurate personalization of OBEs .49
7.2.11 ET-CM-EL-1 EL — Correct exception list generation rate .50
7.2.12 ET-CM-EL-2 EL — Incorrect exception list generation rate . 51
7.2.13 ETCMPC1 PC — Correct charging rate . 51
7.2.14 ETCMPC2 PC — Overcharging Rate . 52
7.2.15 ETCMPC3 PC — Undercharging rate . 53
7.2.16 ET-CM-PC-4 PC — Latency — TC .54
7.2.17 ET-CM-PC-5 PC — Late payment claims rate . 55
7.2.18 ET-CM-PC-6 PC — Rejected payment claim rate . 55
7.2.19 ETCMBD1 BD — Correct charging rate .56
7.2.20 ETCMBD2 BD — Overcharging rate. 57
7.2.21 ETCMBD3 BD — Undercharging rate .58
7.2.22 ETCMBD4 BD — Incorrect charging rate .58
7.2.23 ET-CM-BD-5 BD — Latency — TC . 59
7.2.24 ETCMBD6 BD — Late billing details rate .60
7.2.25 ETCMBD7 BD — Rejected billing details rate .60
7.2.26 ET-CM-BD-8 BD — Incorrectly rejected billing details rate. 61
7.2.27 ETCMBD9 BD — Inferred billing details rate . 61
7.2.28 ET-CM-CR-1 CR— Usage evidence availability . 62
7.2.29 ET-CM-CR-2 CR— Usage evidence integrity . 62
7.2.30 ET-CM-CR-3 CR- Usage evidence time-to-first-fix .63
7.3 Technology-dependent tests .64
7.3.1 Autonomous discrete specific examination tests .64
7.3.2 Discrete — Optional toll declaration metrics .74
7.3.3 Autonomous continuous specific examination tests .77
Annex A (informative) Examination test documentation template .88
Annex B (informative) Examination framework considerations .89
Annex C (informative) Statistical considerations .92
Annex D (informative) Methods for reducing sample sizes during the evaluation phase .97
Annex E (informative) Examples of specific examination frameworks . 101
Annex F (informative) Defining performance requirements . 112
Bibliography . 120
iv
© ISO 2022 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/DTS 37444:2022(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards
bodies (ISO member bodies). The work of preparing International Standards is normally carried out
through ISO technical committees. Each member body interested in a subject for which a technical
committee has been established has the right to be represented on that committee. International
organizations, governmental and non-governmental, in liaison with ISO, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO documents should be noted. This document was drafted in accordance with the
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO shall not be held responsible for identifying any or all such patent rights. Details of
any patent rights identified during the development of the document will be in the Introduction and/or
on the ISO list of patent declarations received (see www.iso.org/patents).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see
www.iso.org/iso/foreword.html.
This document was prepared by Technical Committee ISO/TC 204, Intelligent transport systems, in
collaboration with the European Committee for Standardization (CEN) Technical Committee CEN/TC
278, Intelligent transport systems, in accordance with the Agreement on technical cooperation between
ISO and CEN (Vienna Agreement).
This first edition cancels and replaces the second editions ISO/TS 17444-1:2017 and ISO/TS 17444-2:2017,
which have been technically revised.
The main changes are as follows:
— the resulting document has been renumbered as ISO/TS 37444;
— various editorial changes have been made to improve the readability of the text;
— a technology-neutral definition of metrics and examination tests has been applied, which also includes
support for tolling systems based on automatic number plate recognition (ANPR) technology;
— terminology and references to other documents have been updated.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.
v
© ISO 2022 – All rights reserved
---------------------- Page: 5 ----------------------
ISO/DTS 37444:2022(E)
Introduction
0.1 General
Electronic tolling systems are complex distributed systems involving critical technology such as
dedicated short-range communication (DSRC), camera-based technology (e.g. automatic number plate
recognition, ANPR) and global navigation satellite systems (GNSS). These technologies are all subject
to a certain behaviour that can affect the computation of the charges. Thus, to protect the interests
of the different stakeholders involved, particularly toll service users (SUs) and toll chargers (TCs), it
is essential to define metrics that measure the performance of the system in terms of computation
of charges, and that ensure that the potential resulting errors are acceptable. These metrics will be
a useful tool for establishing requirements for the systems and for examining the system capabilities
during acceptance and throughout the operational life of the system.
In addition, to ensure the interoperability of different systems, it is necessary to agree on common
metrics for use and on the actual values that define the required acceptable performances. However,
these points are not covered in this document.
Instead, this document is defined as a toolbox standard of examination tests, with a method for defining
and documenting a specific examination framework to meet specific needs. The detailed choice of the set
of examination tests within an examination framework depends on the application and the respective
context. Conformance to this document means using the definitions and prescriptions laid out in this
document whenever the respective system aspects are subjected to performance measurements, rather
than using other definitions and examination methods.
0.2 Charging performance metrics
This documents also defines a set of charging performance metrics with definitions, principles and
formulations, which together make up a reference framework for the establishment of requirements for
electronic fee collection (EFC) systems and their later examination of the charging performance.
These charging performance metrics are intended for use with any toll scheme, regardless of its
technical underpinnings, system architecture, tariff structure, geographical coverage or organizational
model. They are defined to treat technical details that can differ between technologies as a “black
box”. They focus solely on the outcome of the charging process, i.e. the amount charged in relation
to a pre-measured or theoretically correct amount, rather than intermediate variables from various
components as sensors, such as positioning accuracy, signal range or optical resolution. This approach
ensures comparable results for each metric in all relevant situations.
The metrics are designed to cover the information exchanged on the front-end (FE) interface and the
interoperability interfaces between toll service providers (TSPs) and TCs, as well as information on the
endtoend level.
Metrics for the following information exchanges are defined:
— charge reports (including usage evidence);
— toll declarations;
— billing details and associated event data;
— payment claims on the level of user accounts;
— exception lists;
— endtoend metrics which assess the overall performance of the charging process.
The proposed metrics are specifically addressed to protect the interests of the actors in a toll system,
such as TSPs, TCs and SUs. They can be used to define requirements (e.g. for requests for proposals) and
for performance assessment.
vi
© ISO 2022 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/DTS 37444:2022(E)
Toll schemes take on various forms as identified in ISO 17573-1 and ISO 12855. In order to create a
uniform performance metric specification, toll schemes are grouped into two classes based on the
character of their primary charging variable:
— charging based on discrete events (charges associated to the fact that a vehicle is crossing or
standing within a certain zone);
— charging based on a continuous measurement (duration or distance).
The following are examples of discrete (event-based) toll schemes.
— Single object charging: a road section, bypass, bridge, tunnel, mountain pass or even a ferry, charged
per passage.
EXAMPLE 1 Most tolled bridges belong to this category.
— Closed road charging: a fixed amount is charged for a certain combination of entry and exit on a
motorway or other closed road network.
EXAMPLE 2 Many of the motorways in Southern Europe belong to this category.
— Discrete road links charging: determined by use of specified road links, whether or not they are
used in their entirety.
EXAMPLE 3 Heavy goods vehicle (HGV) charge in Germany.
— Charging for cordon crossing: triggered by passing in or out through a cordon that encircles a city
core, for example.
EXAMPLE 4 Congestion and infrastructure charging schemes in Stockholm and Gothenburg (Sweden).
The following are examples of continuous toll schemes.
— Charging based on direct distance measurement: defined as an amount per km driven.
EXAMPLE 5 HGV charge in Switzerland and US basic vehicle miles travelled toll systems concepts.
— Charging based on direct distance measurement in different tariff zones or road types: defined
as an amount per km driven, with different tariffs applying in different zones or on different road
types. This is a widely discussed approach, also known as time-distance-place charging, and is
under consideration in European countries.
EXAMPLE 6 OReGO, the pilot programme in Oregon, North America.
— Time in use charge: determined by the accumulated time a vehicle has been in operation, or
alternatively, by the time the vehicle has been present inside a predefined zone.
In all of these toll schemes, tolls can additionally vary as a function of vehicle class characteristics (such
as trailer presence, number of axles, taxation class and operating function) and depending on time of
day or day of week, such that, for example, tariffs are higher in rush hour and lower on weekends.
With this degree of complexity, it is not surprising to find that the attempts to evaluate and compare
technical solutions for SU charging have been made on an individual basis each time a procurement
or study is initiated, and with only limited ability to reuse prior comparisons made by other testing
entities.
The identification of different types of schemes as proposed in the ISO 17575 series and their grouping
in the mentioned two classes is described in Table 1. Table 1 also identifies the examples mentioned
above.
vii
© ISO 2022 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/DTS 37444:2022(E)
Table 1 — Toll scheme designs grouped according to scheme categories
Examples Scheme type ISO 17575 series category
Single object charging Discrete Sectioned roads pricing
Closed road charging Discrete Sectioned roads pricing
Discrete road links charging Discrete Sectioned roads pricing
Charging for cordon crossing Discrete Cordon pricing
Time in use charge Continuous Area pricing — time
Cumulative distance charge Continuous Area pricing — distance
Charging for cumulative distance in different zones (or by Continuous Area pricing — distance
road type)
0.3 Examination framework
The examination framework that is defined in this document is designed for measuring the metrics
defined in Clause 6. The general aim is to achieve a maximum comparability and reproducibility
of the results without restricting the technological choices in system design. Specific examination
frameworks can be defined for the evaluation and monitoring phases of a project due to the differences
in the availability of equipped vehicles.
a) Evaluation phase
The evaluation phase encompasses system evaluation and selection, as well as commissioning and
rampup during implementation. Important aspects of this phase are:
— relatively small sample sizes; and
— well controlled behaviour of test vehicles.
b) Monitoring phase
After the system has gone into operation, its behaviour needs to be monitored for several reasons,
such as fine-tuning of the system performance, monitoring of service level agreements (SLAs) between
contractual partners (supplier, TC, TSP, etc.). In this phase, the following system aspects can be
expected:
— very large sample sizes possible, but with unknown behaviour of the vehicles;
— in principle all measurements from implementation phase possible, too.
0.4 Reader's guide
To understand the content of this document, the reader should be aware of the methodology and
assumptions used to develop the examination framework; therefore, a suggested reading order is given
below.
a) Annex B provides details of the underlying considerations for developing the examination
framework.
b) Annex C provides background statistical information which will enable the reader to determine
sample sizes and confidence limits based on the defined performance requirements.
c) Clause 5 provides the definition of the examination framework for the evaluation of charging
performance.
d) Clause 6 provides definitions of charging metrics and their a
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.