XML Hive Serde Extract timestamp Hadoop

Question

I am trying to extract the timestamp from an xml using xml serde in Hive. The external table creation links to a hdfs directory. Currently, the timestamp value shows up as null in my table.

I am thinking the timestamp needs to be casted? I am not sure. The rest of the xml information is working correctly and shows up in hive.

Input file is:

<example>
<date>2017-02-09 22:03:58<date>
</example>

Hive Creation Script:

create external table example (
date timestamp
)
ROW FORMAT SERDE 'com.ibm.spss.hive.serde2.xml.XmlSerDe'
WITH SERDEPROPERTIES (
"column.xpath.date"="/example/date/text()"
)
STORED AS
INPUTFORMAT 'com.ibm.spss.hive.serde2.xml.XmlInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat'
LOCATION 'mypath'
TBLPROPERTIES (
"xmlinput.start"="<example>",
"xmlinput.end"="</example>"
);

score 1 · Accepted Answer · answered Feb 11 '17 at 07:26

It seems that only Java primitive types are being supported.
Check out the getPrimitiveValue method in the XmlUtils.java file.

/**
 * (c) Copyright IBM Corp. 2013. All rights reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License").
 * You may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
*/

package com.ibm.spss.hive.serde2.xml.processor;

import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory;

/**
 * The XML utilities
 */
public class XmlUtils {

    /**
     * Private constructor
     */
    private XmlUtils() {
    }

    /**
     * Converts the string value to the java object for the given primitive category
     * 
     * @param value
     *            the value
     * @param primitiveCategory
     *            the primitive category
     * @return the java object
     */
    public static Object getPrimitiveValue(String value, PrimitiveCategory primitiveCategory) {
        if (value != null) {
            try {
                switch (primitiveCategory) {
                    case BOOLEAN:
                        return Boolean.valueOf(value);
                    case BYTE:
                        return Byte.valueOf(value);
                    case DOUBLE:
                        return Double.valueOf(value);
                    case FLOAT:
                        return Float.valueOf(value);
                    case INT:
                        return Integer.valueOf(value);
                    case LONG:
                        return Long.valueOf(value);
                    case SHORT:
                        return Short.valueOf(value);
                    case STRING:
                        return value;
                    default:
                        throw new IllegalStateException(primitiveCategory.toString());
                }
            } catch (Exception ignored) {
            }
        }
        return null;
    }

}

XML Hive Serde Extract timestamp Hadoop

1 Answers1