1

Unfortunately, my code is not shareable since it is for a company project, but I'm completely lost with this error. I've uninstalled and reinstalled data.table and still no fix. The weird thing is that I would think this error would occur locally as well, but it doesn't! data.table works fine locally. I'd like to have a reproducible example, but I can't figure one out because it works locally!

EDIT: Okay I guess what is happening is the following

###Coerce to factors for clean 2x2's.
a_tab <- factor(data_fin$a_result_final,levels = c("Positive","Negative")) 
t_tab <- factor(data_fin$t_result_final,levels = c("Positive","Negative"))
p_tab <- factor(data_fin$p_result_final,levels = c("Positive","Negative"))

tdf <- data.frame(a_tab,c_tab,p_tab)

#Create contingency tables
table <- table(tdf$a_tab,tdf$c_tab,dnn = c("Test 1","Test 2")) %>%
  addmargins(margin = c(1,2),FUN = sum)
print('made B Table')
table_2 <- table(tdf$a_tab,tdf$c_tab,dnn = c("Test 1","Test 3")) %>%
  addmargins(margin = c(1,2),FUN = sum)
print('made C Table')

Warning:Error: object 'CcopyNamedInList' not found

I can't share the data, but the factors come from a data frame containing several data entries along the lines of 'Positive','Negative', and 'Invalid'. Hopefully this follows better standards of question asking.

EDIT: Okay, so I have been restarting R and running the instructions you have listed `

test.data.table()
getDTthreads(verbose=TRUE):
  omp_get_num_procs()            4
  R_DATATABLE_NUM_PROCS_PERCENT  unset (default 50)
  R_DATATABLE_NUM_THREADS        unset
  omp_get_thread_limit()         2147483647
  omp_get_max_threads()          4
  OMP_THREAD_LIMIT               unset
  OMP_NUM_THREADS                unset
  RestoreAfterFork               true
  data.table is using 2 threads. See ?setDTthreads.
test.data.table() running: C:/Users/jkramp/Documents/R/win-library/3.6/data.table/tests/tests.Rraw.bz2 
Running test id 2120.08        
10 longest running tests took 56s (33% of 168s)
      ID time nTest
 1: 1438 8.58   738
 2: 1835 8.38     1
 3: 1648 6.17    91
 4: 1652 5.74    91
 5: 1650 5.71    91
 6: 1223 5.39   728
 7: 1848 4.39     1
 8: 1644 4.30    91
 9: 1642 4.19    91
10: 1646 4.12    91
endian==little, sizeof(long double)==16, sizeof(pointer)==8, TZ=America/Los_Angeles, locale='LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252', l10n_info()='MBCS=FALSE; UTF-8=FALSE; Latin-1=TRUE; codepage=1252', getDTthreads()='omp_get_num_procs()==4; R_DATATABLE_NUM_PROCS_PERCENT==unset (default 50); R_DATATABLE_NUM_THREADS==unset; omp_get_thread_limit()==2147483647; omp_get_max_threads()==4; OMP_THREAD_LIMIT==unset; OMP_NUM_THREADS==unset; RestoreAfterFork==true; data.table is using 2 threads. See ?setDTthreads.'

All 9643 tests in tests/tests.Rraw.bz2 completed ok in 00:02:48 elapsed (00:01:50 cpu) on Fri Oct 25 14:22:24 2019

install.packages('expss')
WARNING: Rtools is required to build R packages but is not currently installed. Please download and install the appropriate version of Rtools before proceeding:

https://cran.rstudio.com/bin/windows/Rtools/
Installing package into ‘C:/Users/jkramp/Documents/R/win-library/3.6’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/bin/windows/contrib/3.6/expss_0.9.1.zip'
Content type 'application/zip' length 1867278 bytes (1.8 MB)
downloaded 1.8 MB

package ‘expss’ successfully unpacked and MD5 sums checked

The downloaded binary packages are in
    C:\Users\jkramp\AppData\Local\Temp\Rtmpy49ttd\downloaded_packages

Everything seems to be installed fine. I also have reinstalled expss. I do get a warning about Rtools, but I think that may be a separate issue? Here is a direct output from my shiny logs:

2019-10-25T21:44:01.961646+00:00 shinyapps[929961]: Warning: Error in data.table: object 'CcopyNamedInList' not found
2019-10-25T21:44:01.966388+00:00 shinyapps[929961]:   67: lapply
2019-10-25T21:44:01.966383+00:00 shinyapps[929961]:   75: data.table
2019-10-25T21:44:01.966388+00:00 shinyapps[929961]:   66: multi_cro
2019-10-25T21:44:01.966384+00:00 shinyapps[929961]:   74: make_datatable_for_cro
2019-10-25T21:44:01.966389+00:00 shinyapps[929961]:   65: cro
2019-10-25T21:44:01.966385+00:00 shinyapps[929961]:   73: elementary_cro
2019-10-25T21:44:01.966386+00:00 shinyapps[929961]:   72: FUN
2019-10-25T21:44:01.966386+00:00 shinyapps[929961]:   71: lapply
2019-10-25T21:44:01.966640+00:00 shinyapps[929961]: Error in data.table(cell_var, col_var, weight) : 
2019-10-25T21:44:01.966641+00:00 shinyapps[929961]:   object 'CcopyNamedInList' not found
2019-10-25T21:44:01.966391+00:00 shinyapps[929961]:   61: server [/srv/connect/apps/1143_r_dashboard/app.R#192]
2019-10-25T21:44:01.966389+00:00 shinyapps[929961]:   64: as.etable.table

The app seems to crash in the middle of using expss, but says that it's a data.table problem. installing and updating expss does not seem to fix my problem on shiny, but again it works locally.

Jdev
  • 110
  • 7
  • 1
    I understand your code is not shareable, but without code it's virtually impossible to help. If you can obfuscate part of the code and data and edit your question with it, then someone could eventually help. But as it stands now, your question is bound to be downvoted or closed due to being "too broad". – PavoDive Oct 24 '19 at 18:06
  • 1
    `CcopyNamedInList` is an internal data.table object that was removed in a recent upgrade. Are you using `:::` to get to the internal object? Search your entire code tree for `:::`. Try `test.data.table()` on your server. If that doesn't work, then you have a faulty installation. data.table checks the R code version matches the .so/.dll when it starts up. Please report your operating system. Please shutdown all windows processes and reinstall cleanly. – Matt Dowle Oct 24 '19 at 23:46
  • What's this shiny `cro` output? Maybe it's something to do with Shiny. Can you ask them to see if they know? What's on the line it points to? /srv/connect/apps/1143_r_dashboard/app.R#192 – Matt Dowle Oct 26 '19 at 04:45
  • Shiny has a server that you have to restart, or something? – Matt Dowle Oct 26 '19 at 04:46

2 Answers2

4

Please study this section of Installation guide: https://github.com/Rdatatable/data.table/wiki/Installation#windows.

That contains:

On Windows, when upgrading any package that uses compiled code, it appears to be important to close all R sessions before upgrading. This releases all locks that Windows holds on dlls. To be really sure, reboot too. Then open a new fresh R session. Some users have an .Rprofile file that automatically loads data.table whenever any R sessions starts. You probably do not have an .Rprofile file but if you do, and it loads any packages (which themselves might load data.table), either temporarily rename .Rprofile or pass --vanilla when starting R to prevent it from running .Rprofile. Once you have a single fresh R session without data.table loaded proceed to installation.

To test your installation
require(data.table)
test.data.table()
If test.data.table() fails then you know the installation has not worked properly. Typical problems on Windows involve functions missing or parameters mismatching. These problems can result in a crash. Reboot your Windows machine to clear all dll locks and reinstall the .zip again. test.data.table() is a reliable way to test your installation. The full suite of over 5,000 tests runs locally on your machine and should not take more than 2 minutes.

Where "Typical problems on Windows involve functions missing or parameters mismatching" is in the same area as the error you've seen: Error: object 'CcopyNamedInList' not found

Also, see note 1 of v1.12.0 from Jan 2019 :

When data.table loads it now checks its DLL version against the version of its R level code. This is to detect installation issues on Windows when i) the DLL is in use by another R session and ii) the CRAN source version > CRAN binary binary which happens just after a new release (R prompts users to install from source until the CRAN binary is available). This situation can lead to a state where the package’s new R code calls old C code in the old DLL; R#17478, #3056. This broken state can persist until, hopefully, you experience a strange error caused by the mismatch. Otherwise, wrong results may occur silently. This situation applies to any R package with compiled code not just data.table, is Windows-only, and is long-standing. It has only recently been understood as it typically only occurs during the few days after each new release until binaries are available on CRAN.

So I did my best to catch problems like this on Windows. data.table is the only package on CRAN which checks itself, as far as I know. It checks itself when it loads. All I can think is that when you write that it doesn't work on the server, what you mean is that you have an R session there that has been open for many days or weeks and you haven't restarted it. That's the only way I can think this could escape the R/dll version match check.

When you investigate this problem, can you give me as much detail as possible please as to when you upgraded, how exactly you upgraded (from source or from binary), etc. I'm assuming you're on Windows, because that's the main platform where problems like this occur due to DLL locking. Please always include sessionInfo() in reports.

Community
  • 1
  • 1
Matt Dowle
  • 58,872
  • 22
  • 166
  • 224
  • 1
    It is not only on Windows. I have an issues like this https://github.com/gdemin/expss/issues/42 quite regularly after new releases of the data.table. As you can see on the first link, the error rises on MacOs. Personally I have encountered it under Linux. The bug is very difficult to reproduce but reinstalling both data.table and expss helps in the most cases. – Gregory Demin Oct 25 '19 at 08:32
  • Please see my new edit containing more error information and the result of test.data.table(). Am I missing anything? – Jdev Oct 25 '19 at 21:58
  • @Jdev Maybe it's something to do with Shiny. Can you ask them to see if they know? What's on the line it points to? /srv/connect/apps/1143_r_dashboard/app.R#192 – Matt Dowle Oct 26 '19 at 04:35
  • @GregoryDemin That is of concern. But I don't have enough to go on. Have you seen the version checks I've put into data.table that check DLL and R code? How on earth can it still be happening then? I need your help to provide more information as to how you upgrade. Whenever I upgrade I'm always careful to close all R sessions first, and just upgrade using one new one. Problems arise when you don't do that. Could that be the problem? – Matt Dowle Oct 26 '19 at 04:42
  • @MattDowle Result of the `test.data.table()` - https://github.com/gdemin/expss/issues/42#issuecomment-547046784. Perhaps, I do something wrong inside expss but definitely I don't call internal data.table functions. – Gregory Demin Oct 28 '19 at 22:09
0

It seems to be a bug with the expss package when deployed to shiny. Shiny support is working to find an answer, but I ended up rewriting some chunks of my code to replace expss with alternative functions from the shiny package and base R to resolve the issue. Thank you all for your input, but I can't spend anymore time investigating the issue because I have a deadline. I guess I'll have to make contingency tables for Excel downloads some other way.

Jdev
  • 110
  • 7
  • I am author of the expss package. I use it with Shiny a lot in rather complex applications and I didn't see any bugs like this. – Gregory Demin Oct 28 '19 at 22:02
  • Moreover, I have almost 100% test coverage and there are no errors on CRAN with these tests - https://cran.r-project.org/web/checks/check_results_expss.html – Gregory Demin Oct 28 '19 at 22:11