--- title: "Upsilon test by example" author: - name: Xuye Luo and Mingzhou Song, New Mexico State University date: December 20, 2025 output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Upsilon test by example} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} bibliography: references.bib csl: apa.csl --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) y.lab <- expression(- log[10]~italic(p)*plain(-value)) # "-Log10(p-value)" ``` Here, we illustrate how the Upsilon test promote *dominant* function patterns among categorical variables, while most other tests of association favor *all* function patterns. This property makes the Upsilon test favor robust association patterns. We explain both differences and similarities between the Upsilon test and four other tests of association on the same contingency tables. They include three long established tests: Pearson's chi-squared test [@pearson1900], Woolf's *G*-Test [@woolf1957], and Fisher's exact test [@fisher1935], and the recent *U*-statistic permutation (USP) test [@berrett2020; @berrett2021]. The Upsilon test runs as fast as Pearson's chi-squared test and *G*-Test. Fisher's exact test is slow due to table enumeration. The USP test may consume many more CPU cycles than others due to the number of permutations required for *p*-value precision. ```{=html} ``` ```{r include=FALSE} require(Upsilon) require(ggplot2) library(patchwork) #require(gtools) require(FunChisq) require(DescTools) require(USP) require(metan) methods_level = c("Upsilon","Pearson's chi-squared","Fisher's Exact","G-Test","USP") ``` ## Promoting dominant function patterns The Upsilon test promotes *non-constant* functions that dominate counts in a contingency table, while most other methods promote all mathematical functions. Here, a contingency table is formed by observed frequencies of two categorical variables: one as the row index and the other as the column index. **Table 1** contains a perfect function pattern tested significant by all methods at $\alpha=0.05$. The non-constant function pattern is covered by the entire 3 $\times$ 3 table, thus dominating the table. **Table 2** also contains a perfect function pattern spanning the entire table. However, the table is dominated by the single entry of 16, which can be considered a constant function. The *p*-value by the Upsilon test is 0.4 demoting the table, while in sharp contrast all other tests call the table significant at much smaller *p*-values. ```{r echo=FALSE, fig.width=12, fig.height=12, out.width="90%", fig.align='center', warning=FALSE, message=FALSE} T1 = matrix(c(6,0,0, 0,6,0, 0,0,6 ), nrow = 3, byrow = T) upsilon_pvalue1 = as.numeric(upsilon.test(T1)$p.value) chisq_pvalue1 = as.numeric(modified.chisq.test(T1)$p.value) fisher_pvalue1 = fisher.test(T1)$p.value gtest_pvalue1 = as.numeric(modified.gtest(T1)$p.value) usp_pvalue1 = USP.test(T1)$p.value marginal_pvalue1 = data.frame("p.value" = c(upsilon_pvalue1,chisq_pvalue1,fisher_pvalue1,gtest_pvalue1,usp_pvalue1)) marginal_pvalue1$Method = methods_level marginal_pvalue1$table = ifelse(marginal_pvalue1$p.value <= 0.05, "Significant", "Insignificant") marginal_pvalue1$table = factor(marginal_pvalue1$table, levels = c("Significant", "Insignificant")) marginal_pvalue1$log.p.value = -log10(marginal_pvalue1$p.value) y_max = max(marginal_pvalue1$log.p.value) T2 = matrix(c(16,0,0, 0,1,0, 0,0,1), nrow = 3, byrow = T) upsilon_pvalue2 = as.numeric(upsilon.test(T2)$p.value) chisq_pvalue2 = as.numeric(modified.chisq.test(T2)$p.value) fisher_pvalue2 = fisher.test(T2)$p.value gtest_pvalue2 = as.numeric(modified.gtest(T2)$p.value) usp_pvalue2 = USP.test(T2)$p.value marginal_pvalue2 = data.frame("p.value" = c(upsilon_pvalue2,chisq_pvalue2,fisher_pvalue2,gtest_pvalue2,usp_pvalue2)) marginal_pvalue2$Method = methods_level marginal_pvalue2$table = ifelse(marginal_pvalue2$p.value <= 0.05, "Significant", "Insignificant") marginal_pvalue2$table = factor(marginal_pvalue2$table, levels = c("Significant", "Insignificant")) marginal_pvalue2$log.p.value = -log10(marginal_pvalue2$p.value) p1_table <- plot_matrix(T1, title = "Table 1. Dominant function", shape.color = "palegreen", size.by = "none",x.axis = "",y.axis = "", number.size = 8) + theme(plot.title = element_text(size = 22, face = "bold")) p2_table <- plot_matrix( T2, title = "Table 2. Dominant constant function", shape.color = "lavender", size.by = "none", x.axis = "", y.axis = "", number.size = 8) + theme(plot.title = element_text(size = 22, face = "bold")) p3_bar <- ggplot(data = marginal_pvalue1, aes(x = factor(Method, levels = methods_level), y = log.p.value, fill = table)) + geom_bar(stat = "identity", position = "dodge", color = "white") + scale_y_continuous(trans = "sqrt", name = y.lab, limits = c(0,y_max)) + #geom_hline(yintercept = -log10(0.05), linetype = "dashed", color = "black")+ labs(title = NULL, x = NULL, y = y.lab, fill = "") + scale_fill_manual(values = c("Significant" = "lightblue1", "Insignificant" = "lightpink")) + geom_text(aes(label = format(p.value, scientific = TRUE, digits = 1), vjust = ifelse(log.p.value < 1.0, -0.5, 1.5)), size=6, position = position_dodge(width = 0.9)) + theme(axis.text.x = element_text(angle = 15, hjust = 1, face = "bold", size = 16), axis.text.y = element_text(size = 15), axis.title.y = element_text(size = 20), legend.position = "top", legend.key.size = unit(1.5, "lines"), legend.text = element_text(size = 25), aspect.ratio = 1) p4_bar <- ggplot(data = marginal_pvalue2, aes(x = factor(Method, levels = methods_level), y = log.p.value, fill = table)) + geom_bar(stat = "identity", position = "dodge", color = "white") + scale_y_continuous(trans = "sqrt", name = y.lab, limits = c(0,y_max)) + #geom_hline(yintercept = -log10(0.05), linetype = "dashed", color = "black")+ labs(title = NULL, x = NULL, y = y.lab, fill = "") + scale_fill_manual(values = c("Significant" = "lightblue1", "Insignificant" = "lightpink")) + geom_text(aes(label = format(p.value, scientific = TRUE, digits = 1), vjust = ifelse(log.p.value < 1.0, -0.5, 1.5)), size=6, position = position_dodge(width = 0.9)) + theme(axis.text.x = element_text(angle = 15, hjust = 1, face = "bold", size = 16), axis.text.y = element_text(size = 15), axis.title.y = element_text(size = 20), legend.position = "top", legend.key.size = unit(1.5, "lines"),legend.text = element_text(size = 25), aspect.ratio = 1) final_plot <- (p1_table / p3_bar) | (p2_table / p4_bar) final_plot ``` > In the *p*-value bar plots hereafter, the $y$-axis is $-\log_{10}$ *p*-value, with the original *p*-values printed at the top of bar. ## Demoting non-dominant function patterns **Table 3** contains a function pattern dominant in the top-left 2 $\times$ 2 sub-table. All tests declared this table significant. The function is not perfect but strong, where the entries containing 1 can be considered noise. **Table 4** contains the same 2 $\times$ 2 sub-table which is no longer dominant. The bottom row of the table dominates the count and presents a constant function pattern. The Upsilon test gave an insignificant *p*-value of 0.4, while all other tests returned substantially lower *p*-values calling the pattern significant. ```{r echo =FALSE, fig.width=12, fig.height=12, out.width="90%", fig.align='center', warning=FALSE, message=FALSE} T1 = matrix(c(10,1,1, 1,10,1, 1,1,1), nrow = 3, byrow = T) upsilon_pvalue1 = as.numeric(upsilon.test(T1)$p.value) chisq_pvalue1 = as.numeric(modified.chisq.test(T1)$p.value) fisher_pvalue1 = fisher.test(T1)$p.value gtest_pvalue1 = as.numeric(modified.gtest(T1)$p.value) usp_pvalue1 = USP.test(T1)$p.value marginal_pvalue1 = data.frame("p.value" = c(upsilon_pvalue1,chisq_pvalue1,fisher_pvalue1,gtest_pvalue1,usp_pvalue1)) marginal_pvalue1$Method = methods_level marginal_pvalue1$table = ifelse(marginal_pvalue1$p.value <= 0.05, "Significant", "Insignificant") marginal_pvalue1$table = factor(marginal_pvalue1$table, levels = c("Significant", "Insignificant")) marginal_pvalue1$log.p.value = -log10(marginal_pvalue1$p.value) y_max = max(marginal_pvalue1$log.p.value) T2 = matrix(c(10,1,1, 1,10,1, 100,100,100), nrow = 3, byrow = T) upsilon_pvalue2 = as.numeric(upsilon.test(T2)$p.value) chisq_pvalue2 = as.numeric(modified.chisq.test(T2)$p.value) fisher_pvalue2 = fisher.test(T2)$p.value gtest_pvalue2 = as.numeric(modified.gtest(T2)$p.value) usp_pvalue2 = USP.test(T2)$p.value marginal_pvalue2 = data.frame("p.value" = c(upsilon_pvalue2,chisq_pvalue2,fisher_pvalue2,gtest_pvalue2,usp_pvalue2)) marginal_pvalue2$Method = methods_level marginal_pvalue2$table = ifelse(marginal_pvalue2$p.value <= 0.05, "Significant", "Insignificant") marginal_pvalue2$table = factor(marginal_pvalue2$table, levels = c("Significant", "Insignificant")) marginal_pvalue2$log.p.value = -log10(marginal_pvalue2$p.value) p1_table <- plot_matrix(T1, title = "Table 3. Dominant function", shape.color = "palegreen", size.by = "none", x.axis = "", y.axis = "", number.size = 8) + theme(plot.title = element_text(size = 22, face = "bold")) p2_table <- plot_matrix(T2, title = "Table 4. Non-dominant function", shape.color = "lavender", size.by = "none", x.axis = "", y.axis = "", number.size = 8) + theme(plot.title = element_text(size = 22, face = "bold")) p3_bar <- ggplot(data = marginal_pvalue1, aes(x = factor(Method, levels = methods_level), y = log.p.value, fill = table)) + geom_bar(stat = "identity", position = "dodge", color = "white") + scale_y_continuous(trans = "sqrt", name = y.lab, limits = c(0, y_max)) + labs(title = NULL, x = NULL, y = y.lab, fill = "") + scale_fill_manual(values = c("Significant" = "lightblue1", "Insignificant" = "lightpink")) + geom_text(aes(label = format(p.value, scientific = TRUE, digits = 1), vjust = ifelse(log.p.value < 1.0, -0.5, 1.5)), size = 6, position = position_dodge(width = 0.9)) + theme_minimal() + theme(axis.text.x = element_text(angle = 15, hjust = 1, face = "bold", size = 16), axis.text.y = element_text(size = 15), axis.title.y = element_text(size = 20), legend.position = "top", legend.key.size = unit(1.5, "lines"), legend.text = element_text(size = 25), aspect.ratio = 1) p4_bar <- ggplot(data = marginal_pvalue2, aes(x = factor(Method, levels = methods_level), y = log.p.value, fill = table)) + geom_bar(stat = "identity", position = "dodge", color = "white") + scale_y_continuous(trans = "sqrt", name = y.lab, limits = c(0, y_max)) + labs(title = NULL, x = NULL, y = y.lab, fill = "") + scale_fill_manual(values = c("Significant" = "lightblue1", "Insignificant" = "lightpink")) + geom_text(aes(label = format(p.value, scientific = TRUE, digits = 1), vjust = ifelse(log.p.value < 1.0, -0.5, 1.5)), size = 6, position = position_dodge(width = 0.9)) + theme_minimal() + theme(axis.text.x = element_text(angle = 15, hjust = 1, face = "bold", size = 16), axis.text.y = element_text(size = 15), axis.title.y = element_text(size = 20), legend.position = "top", legend.key.size = unit(1.5, "lines"), legend.text = element_text(size = 25), aspect.ratio = 1) final_plot <- (p1_table / p3_bar) | (p2_table / p4_bar) final_plot ``` ## Robust to change in low expected count **Tables 5 and 6** are dominated by the 1 $\times$ 2 sub-table on the top-left. Both represent constant function patterns and are declared to be insignificant by the Upsilon test at similar *p*-values close to 1.0. The USP test also declared them insignificant. However, the tables differ in the last column with the entry at the bottom-right corner having low expected counts (given the marginals) of 2/63 and 3/63, respectively. Despite the two tables being highly similar except for the sparse last columns, the remaining tests consider them differ dramatically with **Table 5** being insignificant and **Table 6** being significant. This demonstrates that the Upsilon test is not swayed by the instability of small numbers; a tiny shift in count is insufficient to turn a pattern from non-function to function and vice versa. ```{r echo = FALSE, fig.width=12, fig.height=12, out.width="90%", fig.align='center', warning=FALSE, message=FALSE} library(ggplot2) library(patchwork) library(Upsilon) T1 = matrix(c(30,30,1, 1,1,0), nrow = 2, byrow = T) upsilon_pvalue1 = as.numeric(upsilon.test(T1)$p.value) chisq_pvalue1 = as.numeric(modified.chisq.test(T1)$p.value) fisher_pvalue1 = fisher.test(T1)$p.value gtest_pvalue1 = as.numeric(modified.gtest(T1)$p.value) usp_pvalue1 = USP.test(T1)$p.value marginal_pvalue1 = data.frame("p.value" = c(upsilon_pvalue1,chisq_pvalue1,fisher_pvalue1,gtest_pvalue1,usp_pvalue1)) marginal_pvalue1$Method = methods_level marginal_pvalue1$table = ifelse(marginal_pvalue1$p.value <= 0.05, "Significant", "Insignificant") marginal_pvalue1$table = factor(marginal_pvalue1$table, levels = c("Significant", "Insignificant")) marginal_pvalue1$log.p.value = -log10(marginal_pvalue1$p.value) T2 = matrix(c(30,30,0, 1,1,1), nrow = 2, byrow = T) upsilon_pvalue2 = as.numeric(upsilon.test(T2)$p.value) chisq_pvalue2 = as.numeric(modified.chisq.test(T2)$p.value) fisher_pvalue2 = fisher.test(T2)$p.value gtest_pvalue2 = as.numeric(modified.gtest(T2)$p.value) usp_pvalue2 = USP.test(T2)$p.value marginal_pvalue2 = data.frame("p.value" = c(upsilon_pvalue2,chisq_pvalue2,fisher_pvalue2,gtest_pvalue2,usp_pvalue2)) marginal_pvalue2$Method = methods_level marginal_pvalue2$table = ifelse(marginal_pvalue2$p.value <= 0.05, "Significant", "Insignificant") marginal_pvalue2$table = factor(marginal_pvalue2$table, levels = c("Significant", "Insignificant")) marginal_pvalue2$log.p.value = -log10(marginal_pvalue2$p.value) y_max = max(marginal_pvalue2$log.p.value) p1_table <- plot_matrix(T1, title = "Table 5. Non-function", shape.color = "lavender", size.by = "none", x.axis = "", y.axis = "", number.size = 8) + theme(plot.title = element_text(size = 22, face = "bold")) p2_table <- plot_matrix(T2, title = "Table 6. Non-function", shape.color = "lavender", size.by = "none", x.axis = "", y.axis = "", number.size = 8) + theme(plot.title = element_text(size = 22, face = "bold")) p3_bar <- ggplot(data = marginal_pvalue1, aes(x = factor(Method, levels = methods_level), y = log.p.value, fill = table)) + geom_bar(stat = "identity", position = "dodge", color = "white") + scale_y_continuous(trans = "sqrt", name = y.lab, limits = c(0, y_max)) + labs(title = NULL, x = NULL, y = y.lab, fill = "") + scale_fill_manual(values = c("Significant" = "lightblue1", "Insignificant" = "lightpink")) + geom_text(aes(label = format(p.value, scientific = FALSE, digits = 2), vjust = ifelse(log.p.value < 1.0, -0.5, 1.5)), size = 6, position = position_dodge(width = 0.9)) + theme_minimal() + theme(axis.text.x = element_text(angle = 15, hjust = 1, face = "bold", size = 16), axis.text.y = element_text(size = 15), axis.title.y = element_text(size = 20), legend.position = "top", legend.key.size = unit(1.5, "lines"), legend.text = element_text(size = 25), aspect.ratio = 1) p4_bar <- ggplot(data = marginal_pvalue2, aes(x = factor(Method, levels = methods_level), y = log.p.value, fill = table)) + geom_bar(stat = "identity", position = "dodge", color = "white") + scale_y_continuous(trans = "sqrt", name = y.lab, limits = c(0, y_max)) + labs(title = NULL, x = NULL, y = y.lab, fill = "") + scale_fill_manual(values = c("Significant" = "lightblue1", "Insignificant" = "lightpink")) + geom_text(aes(label = format(p.value, scientific = TRUE, digits = 1), vjust = ifelse(log.p.value < 1.0, -0.5, 1.5)), size = 6, position = position_dodge(width = 0.9)) + theme_minimal() + theme(axis.text.x = element_text(angle = 15, hjust = 1, face = "bold", size = 16), axis.text.y = element_text(size = 15), axis.title.y = element_text(size = 20), legend.position = "top", legend.key.size = unit(1.5, "lines"), legend.text = element_text(size = 25), aspect.ratio = 1) final_plot <- (p1_table / p3_bar) | (p2_table / p4_bar) final_plot ``` ## Lung transplant surgery type and outcome **Table 8** is from a clinical study of lung transplant surgeries [@jung2003] . Columns represent two surgery options A and B; rows represent four possible outcomes from grade G0 to G3. The *p-*value by the Upsilon test is 0.12 ( \> 0.05), giving an insignificant result, the same as the original study. The USP test gave *p*-values ranging from 0.05 to 0.10 run-to-run. However, all remaining tests returned *p-*values smaller than 0.05, contrary to expert intuition in the original study. ```{=html}
Table 8. Lung transplant data
Surgery
A B




Grade
G0 6 0
G1 8 12
G2 8 15
G3 2 1
``` ```{r include=FALSE} T1= matrix(c(6,0, 8,12, 8,15, 2,1),nrow = 4,byrow = T) upsilon_pvalue1 = as.numeric(upsilon.test(T1)$p.value) chisq_pvalue1 = as.numeric(modified.chisq.test(T1)$p.value) fisher_pvalue1 = fisher.test(T1)$p.value gtest_pvalue1 = as.numeric(modified.gtest(T1)$p.value) usp_pvalue1 = USP.test(T1)$p.value marginal_pvalue1 = data.frame("p.value" = c(upsilon_pvalue1,chisq_pvalue1,fisher_pvalue1,gtest_pvalue1,usp_pvalue1)) marginal_pvalue1$Method = methods_level marginal_pvalue1$table = ifelse(marginal_pvalue1$p.value <= 0.05, "Significant", "Insignificant") marginal_pvalue1$table = factor(marginal_pvalue1$table, levels = c("Significant", "Insignificant")) marginal_pvalue1$log.p.value = -log10(marginal_pvalue1$p.value) ``` ```{r echo = FALSE, fig.width=12, fig.height=6, out.width="90%", fig.align='center', warning=FALSE, message=FALSE} y_max = max(marginal_pvalue1$log.p.value) p_bar <- ggplot(data = marginal_pvalue1, aes(x = factor(Method, levels = methods_level), y = log.p.value, fill = table)) + geom_bar(stat = "identity", position = "dodge", color = "white") + # scale_y_continuous(trans = "sqrt", name = y.lab, limits = c(0, y_max)) + labs(title = NULL, x = NULL, y = y.lab, fill = "") + scale_fill_manual(values = c("Significant" = "lightblue1", "Insignificant" = "lightpink")) + geom_text(aes(label = format(p.value, scientific = FALSE, digits = 1), vjust = ifelse(log.p.value < 0.5, -0.5, 1.5)), size=12, position = position_dodge(width = 0.9)) + theme_minimal() + theme(axis.text.x = element_text(angle = 15, hjust = 1, face = "bold", size = 25), axis.text.y = element_text(size = 18), axis.title.y = element_text(size = 25), legend.position = "top", legend.key.size = unit(1.5, "lines"), legend.text = element_text(size = 25)) p_bar ``` ## A contingency table showing similar testing results **Table 7** presents cross classification of party affiliation by gender from the 2018 General Social Survey [@kim2022]. All tests declared a significant association between gender and party affiliation, with the Upsilon test result being the most significant. ```{=html}
Table 7. Party affiliation by gender from 2018 General Social Survey
Democrat Independent Republican
Female 359 133 234
Male 257 96 253
``` ```{r include=FALSE} T1= matrix(c(359,133,234, 257,96,253 ),nrow = 2,byrow = T) upsilon_pvalue1 = as.numeric(upsilon.test(T1)$p.value) chisq_pvalue1 = as.numeric(modified.chisq.test(T1)$p.value) fisher_pvalue1 = fisher.test(T1)$p.value gtest_pvalue1 = as.numeric(modified.gtest(T1)$p.value) usp_pvalue1 = USP.test(T1)$p.value marginal_pvalue1 = data.frame("p.value" = c(upsilon_pvalue1,chisq_pvalue1,fisher_pvalue1,gtest_pvalue1,usp_pvalue1)) marginal_pvalue1$Method = methods_level marginal_pvalue1$table = ifelse(marginal_pvalue1$p.value <= 0.05, "Significant", "Insignificant") marginal_pvalue1$table = factor(marginal_pvalue1$table, levels = c("Significant", "Insignificant")) marginal_pvalue1$log.p.value = -log10(marginal_pvalue1$p.value) ``` ```{r echo = FALSE, fig.width=12, fig.height=6, out.width="90%", fig.align='center', warning=FALSE, message=FALSE} y_max = max(marginal_pvalue1$log.p.value) p_bar <- ggplot(data = marginal_pvalue1, aes(x = factor(Method, levels = methods_level), y = log.p.value, fill = table)) + geom_bar(stat = "identity", position = "dodge", color = "white") + # scale_y_continuous(trans = "sqrt", name = y.lab, limits = c(0, y_max)) + labs(title = NULL, x = NULL, y = y.lab, fill = "") + scale_fill_manual(values = c("Significant" = "lightblue1", "Insignificant" = "lightpink")) + geom_text(aes(label = format(p.value, scientific = FALSE, digits = 1), vjust = ifelse(log.p.value < 1.0, -0.5, 1.5)), size=10, position = position_dodge(width = 0.9)) + theme_minimal() + theme(axis.text.x = element_text(angle = 15, hjust = 1, face = "bold", size = 25), axis.text.y = element_text(size = 18), axis.title.y = element_text(size = 25), legend.position = "top", legend.key.size = unit(1.5, "lines"), legend.text = element_text(size = 25)) p_bar ``` ## How to cite this document Luo, Xuye, & Song, Mingzhou. (2025). Upsilon test by example. Vignettes, *Upsilon: Another Test of Association for Count Data*. R package. ## References