1

Edit: Solution: Upgrading to ISIS 1.17.0 and setting the property isis.persistor.datanucleus.standaloneCollection.bulkLoad=false solved the first two problems.

I am using Apache ISIS 1.16.2 and I try to store Blob/Clob content in a MariaDB database (v10.1.35). Therefore, I use the DB connector org.mariadb.jdbc.mariadb-java-client (v2.3.0) and in the code the @Persistent annotation as shown in many examples and the ISIS documentation.

Using the code below, I just get one single column named content_name (in which the Blob object is serialized in binary form) instead of the three columns content_name, content_mimetype and content_bytes.

This is the Document class with the Blob field content:

@PersistenceCapable(identityType = IdentityType.DATASTORE)
@DatastoreIdentity(strategy = IdGeneratorStrategy.IDENTITY, column = "id")
@DomainObject(editing = Editing.DISABLED, autoCompleteRepository = DocumentRepository.class, objectType = "Document")
@Getter
// ...
public class Document implements Comparable<Document> {

    @Persistent(
        defaultFetchGroup = "false",
        columns = {
            @Column(name = "content_name"),
            @Column(name = "content_mimetype"),
            @Column(name = "content_bytes",
                jdbcType = "BLOB",
                sqlType = "LONGVARBINARY")
            })
    @Nonnull
    @Column(allowsNull = "false")
    @Property(optionality = Optionality.MANDATORY)
    private Blob content;

    // ...

    @Column(allowsNull = "false")
    @Property
    private Date created = new Date();

    public Date defaultCreated() {
        return new Date();
    }

    @Column(allowsNull = "true")
    @Property
    @Setter
    private String owner;

    // ...
}

This creates the following schema for the DomainObject class Document with just one column for the Blob field:

CREATE TABLE `document` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `content_name` mediumblob,
  `created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `owner` varchar(255) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Normally, the class org.apache.isis.objectstore.jdo.datanucleus.valuetypes.IsisBlobMapping of the ISIS framework should do the mapping. But it seems that this Mapper is somehow not involved...

1. Question: How do I get the Blob field being split up in the three columns (as described above and in many demo projects). Even if I switch to HSQLDB, I still get only one column, so this might not be an issue with MariaDB.

2. Question: If I use a Blob/Clob field in a class that inherits from another DomainObject class, I often get a org.datanucleus.exceptions.NucleusException (stack trace see below) and I cannot make head or tail of it. What are potential pitfalls when dealing with inheritance? Why am I getting this exception?

3. Question: I need to store documents belonging to domain objects (as you might have guessed). The proper way of doing so would be to store the documents in a file system tree instead of a database (which also has by default some size limitations for object data) and reference the files in the object. In the Datanucleus documentation I found the extension serializeToFileLocation that should do exactly that. I tried it by adding the line @Extension(vendorName="datanucleus", key="serializeToFileLocation" value="document-repository") to the Blob field, but nothing happened. So my question is: Is this Datanucleus extension compatible with Apache Isis?

If this extension conflicts with Isis, would it be possible to have a javax.jdo.listener.StoreLifecycleListener or org.apache.isis.applib.AbstractSubscriber that stores the Blob on a file system before persisting the domain object to database and restoring it before loading? Are there better solutions available?

That's it for now. Thank you in advance! ;-)

The stack trace to question 2:

... (other Wicket related stack trace)
Caused by: org.datanucleus.exceptions.NucleusException: Creation of SQLExpression for mapping "org.datanucleus.store.rdbms.mapping.java.SerialisedMapping" caused error
        at org.datanucleus.store.rdbms.sql.expression.SQLExpressionFactory.newExpression(SQLExpressionFactory.java:199)
        at org.datanucleus.store.rdbms.sql.expression.SQLExpressionFactory.newExpression(SQLExpressionFactory.java:155)
        at org.datanucleus.store.rdbms.request.LocateBulkRequest.getStatement(LocateBulkRequest.java:158)
        at org.datanucleus.store.rdbms.request.LocateBulkRequest.execute(LocateBulkRequest.java:283)
        at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.locateObjects(RDBMSPersistenceHandler.java:564)
        at org.datanucleus.ExecutionContextImpl.findObjects(ExecutionContextImpl.java:3313)
        at org.datanucleus.api.jdo.JDOPersistenceManager.getObjectsById(JDOPersistenceManager.java:1850)
        at org.apache.isis.core.runtime.system.persistence.PersistenceSession.loadPersistentPojos(PersistenceSession.java:1010)
        at org.apache.isis.core.runtime.system.persistence.PersistenceSession.adaptersFor(PersistenceSession.java:1603)
        at org.apache.isis.core.runtime.system.persistence.PersistenceSession.adaptersFor(PersistenceSession.java:1573)
        at org.apache.isis.viewer.wicket.model.models.EntityCollectionModel$Type$1.loadInBulk(EntityCollectionModel.java:107)
        at org.apache.isis.viewer.wicket.model.models.EntityCollectionModel$Type$1.load(EntityCollectionModel.java:93)
        at org.apache.isis.viewer.wicket.model.models.EntityCollectionModel.load(EntityCollectionModel.java:454)
        at org.apache.isis.viewer.wicket.model.models.EntityCollectionModel.load(EntityCollectionModel.java:70)
        at org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachableModel.java:135)
        at org.apache.isis.viewer.wicket.ui.components.collectioncontents.ajaxtable.CollectionContentsSortableDataProvider.size(CollectionContentsSortableDataProvider.java:68)
        at org.apache.wicket.markup.repeater.data.DataViewBase.internalGetItemCount(DataViewBase.java:142)
        at org.apache.wicket.markup.repeater.AbstractPageableView.getItemCount(AbstractPageableView.java:235)
        at org.apache.wicket.markup.repeater.AbstractPageableView.getRowCount(AbstractPageableView.java:216)
        at org.apache.wicket.markup.repeater.AbstractPageableView.getViewSize(AbstractPageableView.java:314)
        at org.apache.wicket.markup.repeater.AbstractPageableView.getItemModels(AbstractPageableView.java:99)
        at org.apache.wicket.markup.repeater.RefreshingView.onPopulate(RefreshingView.java:93)
        at org.apache.wicket.markup.repeater.AbstractRepeater.onBeforeRender(AbstractRepeater.java:124)
        at org.apache.wicket.markup.repeater.AbstractPageableView.onBeforeRender(AbstractPageableView.java:115)
        at org.apache.wicket.Component.internalBeforeRender(Component.java:950)
        at org.apache.wicket.Component.beforeRender(Component.java:1018)
        at org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1825)
        ... 81 more
 Caused by: org.datanucleus.exceptions.NucleusException: Unable to create SQLExpression for mapping of type "org.datanucleus.store.rdbms.mapping.java.SerialisedMapping" since not supported
        at org.datanucleus.store.rdbms.sql.expression.SQLExpressionFactory#newExpression(SQLExpressionFactory.java:189)
        at org.datanucleus.store.rdbms.sql.expression.SQLExpressionFactory#newExpression(SQLExpressionFactory.java:155)
        at org.datanucleus.store.rdbms.request.LocateBulkRequest#getStatement(LocateBulkRequest.java:158)
        at org.datanucleus.store.rdbms.request.LocateBulkRequest#execute(LocateBulkRequest.java:283)
        at org.datanucleus.store.rdbms.RDBMSPersistenceHandler#locateObjects(RDBMSPersistenceHandler.java:564)
        at org.datanucleus.ExecutionContextImpl#findObjects(ExecutionContextImpl.java:3313)
        at org.datanucleus.api.jdo.JDOPersistenceManager#getObjectsById(JDOPersistenceManager.java:1850)
        at org.apache.isis.core.runtime.system.persistence.PersistenceSession#loadPersistentPojos(PersistenceSession.java:1010)
        at org.apache.isis.core.runtime.system.persistence.PersistenceSession#adaptersFor(PersistenceSession.java:1603)
        at org.apache.isis.core.runtime.system.persistence.PersistenceSession#adaptersFor(PersistenceSession.java:1573)
        at org.apache.isis.viewer.wicket.model.models.EntityCollectionModel$Type$1#loadInBulk(EntityCollectionModel.java:107)
        at org.apache.isis.viewer.wicket.model.models.EntityCollectionModel$Type$1#load(EntityCollectionModel.java:93)
        at org.apache.isis.viewer.wicket.model.models.EntityCollectionModel#load(EntityCollectionModel.java:454)
        at org.apache.isis.viewer.wicket.model.models.EntityCollectionModel#load(EntityCollectionModel.java:70)
        at org.apache.wicket.model.LoadableDetachableModel#getObject(LoadableDetachableModel.java:135)
        at org.apache.isis.viewer.wicket.ui.components.collectioncontents.ajaxtable.CollectionContentsSortableDataProvider#size(CollectionContentsSortableDataProvider.java:68)
        at org.apache.wicket.markup.repeater.data.DataViewBase#internalGetItemCount(DataViewBase.java:142)
        at org.apache.wicket.markup.repeater.AbstractPageableView#getItemCount(AbstractPageableView.java:235)
        at org.apache.wicket.markup.repeater.AbstractPageableView#getRowCount(AbstractPageableView.java:216)
        at org.apache.wicket.markup.repeater.AbstractPageableView#getViewSize(AbstractPageableView.java:314)
        at org.apache.wicket.markup.repeater.AbstractPageableView#getItemModels(AbstractPageableView.java:99)
        at org.apache.wicket.markup.repeater.RefreshingView#onPopulate(RefreshingView.java:93)
        at org.apache.wicket.markup.repeater.AbstractRepeater#onBeforeRender(AbstractRepeater.java:124)
        at org.apache.wicket.markup.repeater.AbstractPageableView#onBeforeRender(AbstractPageableView.java:115)
        // <-- 8 times the following lines
        at org.apache.wicket.Component#internalBeforeRender(Component.java:950)
        at org.apache.wicket.Component#beforeRender(Component.java:1018)
        at org.apache.wicket.MarkupContainer#onBeforeRenderChildren(MarkupContainer.java:1825)
        at org.apache.wicket.Component#onBeforeRender(Component.java:3916)
        // -->
        at org.apache.wicket.Page#onBeforeRender(Page.java:801)
        at org.apache.wicket.Component#internalBeforeRender(Component.java:950)
        at org.apache.wicket.Component#beforeRender(Component.java:1018)
        at org.apache.wicket.Component#internalPrepareForRender(Component.java:2236)
        at org.apache.wicket.Page#internalPrepareForRender(Page.java:242)
        at org.apache.wicket.Component#render(Component.java:2325)
        at org.apache.wicket.Page#renderPage(Page.java:1018)
        at org.apache.wicket.request.handler.render.WebPageRenderer#renderPage(WebPageRenderer.java:124)
        at org.apache.wicket.request.handler.render.WebPageRenderer#respond(WebPageRenderer.java:195)
        at org.apache.wicket.core.request.handler.RenderPageRequestHandler#respond(RenderPageRequestHandler.java:175)
        at org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor#respond(RequestCycle.java:895)
        at org.apache.wicket.request.RequestHandlerStack#execute(RequestHandlerStack.java:64)
        at org.apache.wicket.request.cycle.RequestCycle#execute(RequestCycle.java:265)
        at org.apache.wicket.request.cycle.RequestCycle#processRequest(RequestCycle.java:222)
        at org.apache.wicket.request.cycle.RequestCycle#processRequestAndDetach(RequestCycle.java:293)
        at org.apache.wicket.protocol.http.WicketFilter#processRequestCycle(WicketFilter.java:261)
        at org.apache.wicket.protocol.http.WicketFilter#processRequest(WicketFilter.java:203)
        at org.apache.wicket.protocol.http.WicketFilter#doFilter(WicketFilter.java:284)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain#doFilter(ServletHandler.java:1668)
        at org.apache.isis.core.webapp.diagnostics.IsisLogOnExceptionFilter#doFilter(IsisLogOnExceptionFilter.java:52)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain#doFilter(ServletHandler.java:1668)
        at org.apache.shiro.web.servlet.AbstractShiroFilter#executeChain(AbstractShiroFilter.java:449)
        at org.apache.shiro.web.servlet.AbstractShiroFilter$1#call(AbstractShiroFilter.java:365)
        at org.apache.shiro.subject.support.SubjectCallable#doCall(SubjectCallable.java:90)
        at org.apache.shiro.subject.support.SubjectCallable#call(SubjectCallable.java:83)
        at org.apache.shiro.subject.support.DelegatingSubject#execute(DelegatingSubject.java:383)
        at org.apache.shiro.web.servlet.AbstractShiroFilter#doFilterInternal(AbstractShiroFilter.java:362)
        at org.apache.shiro.web.servlet.OncePerRequestFilter#doFilter(OncePerRequestFilter.java:125)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain#doFilter(ServletHandler.java:1668)
        // ... some Jetty stuff
        at java.lang.Thread#run(Thread.java:748)
JND
  • 65
  • 7
  • A minor update to question 3: The `serializeToFileLocation` Datanucleus extension works as expected. I must have looked in the wrong place... But the Datanucleus extension serializes the Blobs as Java Objects. So I wrote a custom plugin to store the Blob as regular files. – JND Apr 15 '19 at 09:33
  • Please do not edit and add "SOLVED" in the title. That is not how it is done on Stack Overflow. Instead, post your solution as an answer and click the checkmark next to the answer to mark the question as solved. – TT. May 13 '19 at 08:48
  • 1
    Thank you, @TT. I am very new to Stack Overflow. Sorry. – JND May 14 '19 at 09:37
  • No Worries, your answer is appreciated :-) – TT. May 14 '19 at 10:36

1 Answers1

2

After some research, I think that the problem of question 1 and 2 seems to be related to this ISIS bug report #1902.

In short: The datanucleus extension plugin resolving mechanism does not seem to find the ISIS value type adapters and therefore cannot know how to serialize ISIS's Blob/Clob types.

According to the aforementioned ISIS bug report, this problem is fixed in 1.17.0, so I am trying to upgrade from 1.16.2 to this version (which introduced many other problems, but that will be an extra topic).

For question 3 I have found Minio which addresses basically my problem, but it is a bit oversized for my needs. I will keep looking for other solutions to store Blob/Clobs to local file system and will keep Minio in mind...

UPDATE:

  1. I upgraded my project to ISIS version 1.17.0. It solved my problem in question 1 (now I get three columns for a Blob/Clob object).
  2. The problem in question 2 (NucleusException) is not solved by the upgrade. I figured out that it is only thrown if returning a list of DomainObjects with Blob/Clob field(s), i.e. if rendered as standalone table. If I get directly to an object's entity view, no exception is thrown and I can see/modify/download the Blob/Clob content.
  3. In the meantime, I wrote my own datanucleus plugin, that stores the Blobs/Clobs as files on a file system.

UPDATE 2

  1. I found a solution for circumventing the org.datanucleus.exceptions.NucleusException: Unable to create SQLExpression for mapping of type "org.apache.isis.objectstore.jdo.datanucleus.valuetypes.IsisClobMapping" since not supported. It seems to be a problem with bulk-loading (but I do not know any details). Deactivating bulk-load via the property isis.persistor.datanucleus.standaloneCollection.bulkLoad=false (which is initially set to true by ISIS archetypes) solved the problem.
JND
  • 65
  • 7