Title: | Item Response Theory Modeling and Scoring for Judgment Data |
---|---|
Description: | Psychometric analysis and scoring of judgment data using polytomous Item-Response Theory (IRT) models, as described in Myszkowski and Storme (2019) <doi:10.1037/aca0000225> and Myszkowski (2021) <doi:10.1037/aca0000287>. A function is used to automatically compare and select models, as well as to present a variety of model-based statistics. Plotting functions are used to present category curves, as well as information, reliability and standard error functions. |
Authors: | Nils Myszkowski [aut, cre] |
Maintainer: | Nils Myszkowski <[email protected]> |
License: | GPL-3 |
Version: | 1.1.2 |
Built: | 2025-02-02 04:27:54 UTC |
Source: | https://github.com/cran/jrt |
anova method for objects returned by the jrt function.
## S4 method for signature 'jrt' anova(object)
## S4 method for signature 'jrt' anova(object)
object |
An object returned by jrt. |
This function returns the Judge Information Function plot from a jrt
object and the judge number. Information can be plotted as such, as reliability or as standard error. The function may also be used for the information of the entire set of judges. This is a wrapper function and adaptation of the itemplot
function in the package mirt
(Chalmers, 2012). It also uses the plotting functions of the packages directlabels
and ggplot2
.
info.plot( jrt.object, judge = "all", type = "information", title = "auto", column.names = "auto", theta.span = 3.5, y.limits = NULL, y.line = NULL, name.for.y.line = "Threshold", y.line.refers.to.secondary.axis = TRUE, greyscale = FALSE, color.palette = "D3", line.type = 1, line.width = 1, key.width = 3, legend.position = "right", legend.columns = "", theme = "bw", text.size = 10, title.size = text.size + 4, remove.gridlines = TRUE, font.family = "sans", precision = 20, mirt.object.input = F, item = NULL )
info.plot( jrt.object, judge = "all", type = "information", title = "auto", column.names = "auto", theta.span = 3.5, y.limits = NULL, y.line = NULL, name.for.y.line = "Threshold", y.line.refers.to.secondary.axis = TRUE, greyscale = FALSE, color.palette = "D3", line.type = 1, line.width = 1, key.width = 3, legend.position = "right", legend.columns = "", theme = "bw", text.size = 10, title.size = text.size + 4, remove.gridlines = TRUE, font.family = "sans", precision = 20, mirt.object.input = F, item = NULL )
jrt.object |
A object of the |
judge |
A numerical to indicate which judge to plot. Be careful : If a (Generalized) Rating Scale Model was used, then judges may have been removed for the model to be fitted. Provide |
type |
A character to indicate what to plot on the y axis, |
title |
A character title for the plot. By default it is created automatically based on the judge number. |
column.names |
A character to indicate how a judge should be called (Defaults to |
theta.span |
A numeric indicating the maximum |
y.limits |
A numeric vector to manually adjust the minimum and maximum of the y axis (may notably be useful if using reliability). Set to |
y.line |
A numeric to add a (dashed) horizontal line on the plot at the y value indicated (for example for a threshold of acceptable reliability). Defaults to |
name.for.y.line |
A character to indicate how to call the y line in the legend. Default is |
y.line.refers.to.secondary.axis |
A logical to indicate if the y.line should refer to a value on the secondary axis ( |
greyscale |
A logical (default is |
color.palette |
A character value to indicate the colour palette to use. Defaults to |
line.type |
A numeric indicating the line type for the information function curve (default is |
line.width |
A numeric indicating the width for the information function curve (default is |
key.width |
A numeric to indicate the width of the legend key (default is |
legend.position |
A character string or vector of coordinates to position the legend key. Defaults to |
legend.columns |
A numeric to indicate after how many legend key elements to add a line break. Especially useful if using |
theme |
A character value to indicate the background color theme used by |
text.size |
A numeric value to control the size of the text on the plot. |
title.size |
A numeric value to control the size of the plot title (defaults to |
remove.gridlines |
A logical value to remove the gridlines (default is |
font.family |
A character value to control the font family used on the graph. Defaults to |
precision |
A numeric to indicate the degree of precision used to plot the curves. Higher values will increase the accuracy of the graph and make the curves look smoother, but the data generated to plot the graph will be bigger, which will slow down the function. Lower values will do the opposite. Values between |
mirt.object.input |
A logical allowing to input directly an |
item |
For convenience, this argument, more standard to IRT packages, can be used instead of the |
A plot of the information, reliability or standard error function.
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. doi:10.18637/jss.v048.i06
Myszkowski, N., & Storme, M. (2019). Judge Response Theory? A call to upgrade our psychometrical account of creativity judgments. Psychology of Aesthetics, Creativity and the Arts, 13(2), 167-175. doi:10.1037/aca0000225
Myszkowski, N. (2021). Development of the R library “jrt”: Automated item response theory procedures for judgment data and their application with the consensual assessment techniques. Psychology of Aesthetics, Creativity and the Arts, 15(3), 426-438. doi:10.1037/aca0000287
# Load dataset data <- jrt::ratings # Fit model fit <- jrt(data, irt.model = "PCM") # Information function of the first judge info.plot(fit, 1) # Reliability function of the second judge info.plot(fit, 2, type = "reliability") # Standard error function of the entire set of judges info.plot(fit, "all", type = "SE") # See vignette for more options
# Load dataset data <- jrt::ratings # Fit model fit <- jrt(data, irt.model = "PCM") # Information function of the first judge info.plot(fit, 1) # Reliability function of the second judge info.plot(fit, 2, type = "reliability") # Standard error function of the entire set of judges info.plot(fit, "all", type = "SE") # See vignette for more options
This function returns the Judge Category Curves (JCC) plot from a jrt
object and the judge number. This is a wrapper function and adaptation of the itemplot
function in the package mirt
(Chalmers, 2012). It also uses the plotting functions of the packages directlabels
and ggplot2
.
jcc.plot( jrt.object, judge = "all", labelled = T, greyscale = F, vertical.labels = F, title = "auto", column.names = "auto", manual.facet.names = "auto", manual.line.names = "auto", overlay.reliability = F, color.palette = "D3", category.name.for.legend = "", name.for.reliability = "auto", theta.span = 3.5, line.width = 0.8, line.opacity = 1, key.width = 3, legend.position = "right", legend.columns = "", theme = "bw", text.size = 10, title.size = text.size + 4, font.family = "sans", remove.gridlines = T, facet.rows = NULL, facet.cols = NULL, facet.title.position = "top", precision = 20, debug = F, mirt.object.input = F, item = NULL )
jcc.plot( jrt.object, judge = "all", labelled = T, greyscale = F, vertical.labels = F, title = "auto", column.names = "auto", manual.facet.names = "auto", manual.line.names = "auto", overlay.reliability = F, color.palette = "D3", category.name.for.legend = "", name.for.reliability = "auto", theta.span = 3.5, line.width = 0.8, line.opacity = 1, key.width = 3, legend.position = "right", legend.columns = "", theme = "bw", text.size = 10, title.size = text.size + 4, font.family = "sans", remove.gridlines = T, facet.rows = NULL, facet.cols = NULL, facet.title.position = "top", precision = 20, debug = F, mirt.object.input = F, item = NULL )
jrt.object |
A object of the |
judge |
A numerical to indicate which judge(s) to plot. Default is |
labelled |
A logical to indicate whether the curves should be labelled with boxed labels ( |
greyscale |
A logical to indicate whether to plot in greyscale ( |
vertical.labels |
A logical to indicate whether the labels should be vertically oriented ( |
title |
A character title for the plot. By default it is created automatically based on the judge number. |
column.names |
A character to indicate what a column corresponds to (Defaults to |
manual.facet.names |
A vector to indicate the names to give to the different facets. Defaults to |
manual.line.names |
A vector to indicate the individual names to give to the different response categories (or different category curves). Defaults to |
overlay.reliability |
A logical to indicate whether to overlay the reliability function of the item (default is |
color.palette |
A character value to indicate the colour palette to use. Defaults to |
category.name.for.legend |
A character to indicate how to call categories in the legend. Default to |
name.for.reliability |
A character to indicate a preferred name for reliability in the legend or labels. Defaults to |
theta.span |
A numeric indicating the maximum |
line.width |
A numeric indicating the width of the trace lines (default is |
line.opacity |
A numeric vector to indicate opacities for the different category lines. Defaults to |
key.width |
A numeric to indicate the width of the legend key (default is |
legend.position |
A character string or vector of coordinates to position the legend key. Defaults to |
legend.columns |
A numeric to indicate after how many legend key elements to add a line break. Especially useful if using |
theme |
A character value to indicate the background color theme used by |
text.size |
A numeric value to control the size of the text on the plot. |
title.size |
A numeric value to control the size of the plot title (defaults to |
font.family |
A character value to control the font family used on the graph. Defaults to |
remove.gridlines |
A logical value to remove the gridlines (default is |
facet.rows |
A numeric to change the number of rows for faceted plots. Use this one or |
facet.cols |
A numeric to change the number of columns for faceted plots. Use this one or |
facet.title.position |
A character string to indicate the position of the facet titles for faceted plts. Defaults to |
precision |
A numeric to indicate the degree of precision used to plot the category curves. Higher values will increase the accuracy of the graph and make the curves look smoother, but the data generated to plot the graph will be bigger, which will slow down the function. Lower values will do the opposite. Values between |
debug |
A logical to report debug messages (used in development).
Defaults to |
mirt.object.input |
A logical allowing to input directly an |
item |
For convenience, this argument, more standard to IRT packages, can be used instead of the |
A plot of the category curves.
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. doi:10.18637/jss.v048.i06
Myszkowski, N., & Storme, M. (2019). Judge Response Theory? A call to upgrade our psychometrical account of creativity judgments. Psychology of Aesthetics, Creativity and the Arts, 13(2), 167-175. doi:10.1037/aca0000225
Myszkowski, N. (2021). Development of the R library “jrt”: Automated item response theory procedures for judgment data and their application with the consensual assessment techniques. Psychology of Aesthetics, Creativity and the Arts, 15(3), 426-438. doi:10.1037/aca0000287
# Load dataset data <- jrt::ratings # Fit model fit <- jrt(data, irt.model = "PCM") # JCC of the first judge jcc.plot(fit, 1) # See vignette for more options
# Load dataset data <- jrt::ratings # Fit model fit <- jrt(data, irt.model = "PCM") # JCC of the first judge jcc.plot(fit, 1) # See vignette for more options
This function automatically selects appropriate polytomous IRT
models based on an information criterion (e.g. Corrected AIC), then returns
factor scores, standard errors and various IRT psychometric information, as
well as more traditionnal ("CTT") psychometric information. All IRT
estimation procedures are executed with the package mirt
(Chalmers,
2012). The non-IRT procedures use packages psych
and irr
.
jrt( data, irt.model = "auto", summary = T, selection.criterion = "AIC", response.categories = "auto", remove.judges.with.unobserved.categories = F, additional.stats = F, method.factor.scores = "EAP", return.mean.scores = T, prefix.for.outputs = "Judgments", column.names = "Judge", maximum.iterations = 2000, convergence.threshold = 0.001, estimation.algorithm = "EM", status.verbose = F, estimation.package.warnings = F, digits = 3, plots = T, greyscale = F, progress.bar = T, method.item.fit = "X2", select.variables.that.contain = NULL, silent = F, show.calls = F, debug = F )
jrt( data, irt.model = "auto", summary = T, selection.criterion = "AIC", response.categories = "auto", remove.judges.with.unobserved.categories = F, additional.stats = F, method.factor.scores = "EAP", return.mean.scores = T, prefix.for.outputs = "Judgments", column.names = "Judge", maximum.iterations = 2000, convergence.threshold = 0.001, estimation.algorithm = "EM", status.verbose = F, estimation.package.warnings = F, digits = 3, plots = T, greyscale = F, progress.bar = T, method.item.fit = "X2", select.variables.that.contain = NULL, silent = F, show.calls = F, debug = F )
data |
A dataframe or matrix including the judgments to be scored. Note that so far missing data are not supported. This is the only required argument for the function. |
irt.model |
A string value with the name of the model to fit. It can be:
For convenience, models can also be called by their full names
(e.g.
|
summary |
A logical to indicate if summary statistics should be
displayed as messages (default is |
selection.criterion |
A string with the criterion for the automatic
selection. The default is the Akaike Information Criterion
( |
response.categories |
A numeric vector to indicate the possible score
values. For example, use |
remove.judges.with.unobserved.categories |
A logical value to indicate whether to only keep the
judges with all categories observed (based on the |
additional.stats |
A logical to indicate whether to report other
("non-IRT") reliability statistics (based on computations from packages
'psych' and 'irr'). Defaults to |
method.factor.scores |
A string to indicate the method used to compute
the factor scores. Bayesian methods ( |
return.mean.scores |
A logical to indicate whether to return the mean
scores in the output (defaults to |
prefix.for.outputs |
A character used as prefix to name the vectors in the
output data frames. Default is |
column.names |
A character to indicate the preferred name to give to a Judge. Defaults to |
maximum.iterations |
A numeric indicating the maximum number of
iterations used to fit the model (default is |
convergence.threshold |
A numeric to indicate the threshold used to
tolerate convergence (default is |
estimation.algorithm |
A string indicating the estimation algorithm. Can
notably be |
status.verbose |
A logical to indicate whether to output messages
indicating what the package is doing. Defaults to |
estimation.package.warnings |
A logical to indicate whether to output
the warnings and messages of the estimation package. Defaults to
|
digits |
A numeric to indicate the number of digits to round output
statistics by (default is |
plots |
A logical to indicate whether to plot the total information plot
and judge category curves ( |
greyscale |
A logical to indicate whether the plots should be in greyscale ( |
progress.bar |
A logical to indicate whether to show a progress bar
during the automatic model selection. Defaults to |
method.item.fit |
A character value to indicate which fit statistic to use
for the item fit output. Passed to the |
select.variables.that.contain |
A character string to use as data the
variables in the original dataset that contain the string. Based on the
|
silent |
A logical (defaults to |
show.calls |
A logical to report the calls made to fit the different models. This is meant as a didactic options for users who may be interested in switching over to |
debug |
A logical to report debug messages (used in development).
Defaults to |
An object of S4-class jrt
. The factor scores can be accessed
in slot @output.data
.
Chalmers, R., P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48(6), 1-29. doi:10.18637/jss.v048.i06
Myszkowski, N., & Storme, M. (2019). Judge Response Theory? A call to upgrade our psychometrical account of creativity judgments. Psychology of Aesthetics, Creativity and the Arts, 13(2), 167-175. doi:10.1037/aca0000225
Myszkowski, N. (2021). Development of the R library jrt
: Automated item response theory procedures for judgment data and their application with the consensual assessment techniques. Psychology of Aesthetics, Creativity and the Arts, 15(3), 426-438. doi:10.1037/aca0000287
# Load dataset data <- jrt::ratings # Fit models fit <- jrt(data, irt.model = "GRM", # to manually select a model plots = FALSE) # to remove plots # Extract the factor scores [email protected] # In a dataframe with standard errors [email protected] # As a numeric vector # See vignette for more options
# Load dataset data <- jrt::ratings # Fit models fit <- jrt(data, irt.model = "GRM", # to manually select a model plots = FALSE) # to remove plots # Extract the factor scores fit@factor.scores # In a dataframe with standard errors fit@factor.scores.vector # As a numeric vector # See vignette for more options
Object returned by the jrt function.
input.data
The original data
output.data
The output data with factor scores.
fitted.model
The selected model.
response.categories
The count of response categories.
method.factor.scores
The method used to compute factor scores.
imputed.data
The data with imputation.
factor.scores
The factor scores with standard errors as a data.frame.
factor.scores.vector
The factor scores as a vector.
standard.errors.vector
The standard errors as a vector.
mean.scores.vector
The mean scores as a vector.
empirical.reliability
The empirical reliability.
marginal.reliability
The marginal reliability.
item.fit
Tests of item fit.
person.fit
Tests of person fit.
local.dependence
Tests of local dependence.
sample.size
The sample size used in the model.
number.of.judges.in.model
The number of judges (or items) in the model.
column.names
The name used for the columns.
mirt.object
The mirt object of the model.
A simulated dataset with 300 products judged by 6 judges.
ratings
ratings
A data.frame with 300 rows and 6 columns:
Judgments of judge 1
Judgments of judge 2
Judgments of judge 3
Judgments of judge 4
Judgments of judge 5
Judgments of judge 6
A simulated dataset with 350 cases judged by 5 judges, using a planned missingness design.
ratings_missing
ratings_missing
A data.frame with 350 rows (cases) and 5 columns (judges):
Judgments of judge 1
Judgments of judge 2
Judgments of judge 3
Judgments of judge 4
Judgments of judge 5