Class: AgreementMetrics

Metrics for measuring agreement between reviewers/extractors

URI: revaise:AgreementMetrics

```mermaid classDiagram class AgreementMetrics click AgreementMetrics href "../AgreementMetrics/" AgreementMetrics : cohen_kappa

  AgreementMetrics : disagreement_count

  AgreementMetrics : field_level_agreements





    AgreementMetrics --> "*" FieldAgreement : field_level_agreements
    click FieldAgreement href "../FieldAgreement/"



  AgreementMetrics : fleiss_kappa

  AgreementMetrics : krippendorff_alpha

  AgreementMetrics : overall_agreement

  AgreementMetrics : percent_agreement

  AgreementMetrics : total_comparisons

```

Slots

Name	Cardinality and Range	Description	Inheritance
overall_agreement	0..1 Float	Overall agreement rate across all items	direct
cohen_kappa	0..1 Float	Cohen's kappa for inter-rater agreement	direct
fleiss_kappa	0..1 Float	Fleiss' kappa for multiple raters	direct
krippendorff_alpha	0..1 Float	Krippendorff's alpha reliability coefficient	direct
percent_agreement	0..1 Float	Simple percentage agreement	direct
disagreement_count	0..1 Integer	Number of disagreements	direct
total_comparisons	0..1 Integer	Total number of comparisons made	direct
field_level_agreements	* FieldAgreement	Agreement metrics for individual fields	direct

Usages

used by	used in	type	used
ScopingQualityControl	base_agreement_metrics	range	AgreementMetrics
StageQualityControl	base_agreement_metrics	range	AgreementMetrics

Identifier and Mapping Information

Schema Source

• from schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping

Mappings

Mapping Type	Mapped Value
self	revaise:AgreementMetrics
native	revaise:AgreementMetrics

LinkML Source

Direct

name: AgreementMetrics
description: Metrics for measuring agreement between reviewers/extractors
from_schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping
slots:
- overall_agreement
- cohen_kappa
- fleiss_kappa
- krippendorff_alpha
- percent_agreement
- disagreement_count
- total_comparisons
- field_level_agreements
slot_usage:
  overall_agreement:
    name: overall_agreement
    description: Overall agreement rate across all items
    range: float
    minimum_value: 0.0
    maximum_value: 1.0
  cohen_kappa:
    name: cohen_kappa
    description: Cohen's kappa for inter-rater agreement
    range: float
    minimum_value: -1.0
    maximum_value: 1.0
  fleiss_kappa:
    name: fleiss_kappa
    description: Fleiss' kappa for multiple raters
    range: float
    minimum_value: -1.0
    maximum_value: 1.0
  krippendorff_alpha:
    name: krippendorff_alpha
    description: Krippendorff's alpha reliability coefficient
    range: float
    minimum_value: 0.0
    maximum_value: 1.0
  percent_agreement:
    name: percent_agreement
    description: Simple percentage agreement
    range: float
    minimum_value: 0.0
    maximum_value: 100.0
  disagreement_count:
    name: disagreement_count
    description: Number of disagreements
    range: integer
  total_comparisons:
    name: total_comparisons
    description: Total number of comparisons made
    range: integer
  field_level_agreements:
    name: field_level_agreements
    description: Agreement metrics for individual fields
    range: FieldAgreement
    multivalued: true

Induced

name: AgreementMetrics
description: Metrics for measuring agreement between reviewers/extractors
from_schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping
slot_usage:
  overall_agreement:
    name: overall_agreement
    description: Overall agreement rate across all items
    range: float
    minimum_value: 0.0
    maximum_value: 1.0
  cohen_kappa:
    name: cohen_kappa
    description: Cohen's kappa for inter-rater agreement
    range: float
    minimum_value: -1.0
    maximum_value: 1.0
  fleiss_kappa:
    name: fleiss_kappa
    description: Fleiss' kappa for multiple raters
    range: float
    minimum_value: -1.0
    maximum_value: 1.0
  krippendorff_alpha:
    name: krippendorff_alpha
    description: Krippendorff's alpha reliability coefficient
    range: float
    minimum_value: 0.0
    maximum_value: 1.0
  percent_agreement:
    name: percent_agreement
    description: Simple percentage agreement
    range: float
    minimum_value: 0.0
    maximum_value: 100.0
  disagreement_count:
    name: disagreement_count
    description: Number of disagreements
    range: integer
  total_comparisons:
    name: total_comparisons
    description: Total number of comparisons made
    range: integer
  field_level_agreements:
    name: field_level_agreements
    description: Agreement metrics for individual fields
    range: FieldAgreement
    multivalued: true
attributes:
  overall_agreement:
    name: overall_agreement
    description: Overall agreement rate across all items
    from_schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping
    rank: 1000
    owner: AgreementMetrics
    domain_of:
    - AgreementMetrics
    range: float
    minimum_value: 0.0
    maximum_value: 1.0
  cohen_kappa:
    name: cohen_kappa
    description: Cohen's kappa for inter-rater agreement
    from_schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping
    rank: 1000
    owner: AgreementMetrics
    domain_of:
    - AgreementMetrics
    range: float
    minimum_value: -1.0
    maximum_value: 1.0
  fleiss_kappa:
    name: fleiss_kappa
    description: Fleiss' kappa for multiple raters
    from_schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping
    rank: 1000
    owner: AgreementMetrics
    domain_of:
    - AgreementMetrics
    range: float
    minimum_value: -1.0
    maximum_value: 1.0
  krippendorff_alpha:
    name: krippendorff_alpha
    description: Krippendorff's alpha reliability coefficient
    from_schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping
    rank: 1000
    owner: AgreementMetrics
    domain_of:
    - AgreementMetrics
    range: float
    minimum_value: 0.0
    maximum_value: 1.0
  percent_agreement:
    name: percent_agreement
    description: Simple percentage agreement
    from_schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping
    rank: 1000
    owner: AgreementMetrics
    domain_of:
    - AgreementMetrics
    range: float
    minimum_value: 0.0
    maximum_value: 100.0
  disagreement_count:
    name: disagreement_count
    description: Number of disagreements
    from_schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping
    rank: 1000
    owner: AgreementMetrics
    domain_of:
    - AgreementMetrics
    range: integer
  total_comparisons:
    name: total_comparisons
    description: Total number of comparisons made
    from_schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping
    rank: 1000
    owner: AgreementMetrics
    domain_of:
    - AgreementMetrics
    range: integer
  field_level_agreements:
    name: field_level_agreements
    description: Agreement metrics for individual fields
    from_schema: https://open-and-sustainable.github.io/revaise-model/schema/stages/scoping
    rank: 1000
    owner: AgreementMetrics
    domain_of:
    - AgreementMetrics
    range: FieldAgreement
    multivalued: true