3

I've been trying out a couple of ways to load a large json ( > 144 MB) file that I need in my React App using Webpack (v4.3). I was following GitHub Issue on Webpack and tried using a file-loader with Webpack. PFB, the webpack.config.js file.

When I try debugging the value for const ORIGIN_DESTINATION_DATA = require('../.././data/./origin_destination.json'); I see that ORIGIN_DESTINATION_DATA contains "src/data/destination_origin.json" String rather than the actual json data. 

var webpack = require('webpack');
var path = require('path');

const flaskApp = path.join(__dirname, '../', 'app_name');

module.exports = {
 entry: __dirname + '/src/index.jsx',
 output: {
  path: flaskApp + '/static',
  filename: 'bundle.js',
 },
 resolve: {
  extensions: ['.js', '.jsx', '.css', '.json']
 },
 module: {
  rules: [{
    test: /\.(js|jsx)$/,
    exclude: /node_modules/,
    loader: 'babel-loader'
   }, {
    test: /\.less$/,
    loaders: ["style-loader", "css-loader", "less-loader"]
   }, {
    test: /\.css$/,
    loaders: ['style-loader', 'css-loader']
   }, {
    test: /\.(png|woff|woff2|eot|ttf|svg)$/,
    loader: 'url-loader?limit=100000'
   }, {
    test: /\.geojson$/,
    loader: 'url-loader?limit=100000'
   }, {
    type: 'javascript/auto',
    test: /\.json$/,
    use: [{
      loader: 'file-loader',
      options: {
       name: "[path][name].[ext]"
      }
     }
    ]
   }

  ]
 },
};
John
  • 2,445
  • 2
  • 17
  • 25
  • 1
    `file-loader` indeed gives you path to the file in the output folder. That's basically why it is useful to work with non js assets, for example images (usually you don't want to bundle images into you js file). Neither you want to bundle 140M json data file. So you need to manually load json file using say `fetch` and the url comming from the loader. Also I do recommend you to restructure your data somehow. 140Mb data loaded at once sounds like a bad idea. – Yury Tarabanko May 09 '18 at 08:56
  • Thanks @YuryTarabanko going to move the data to Elastic Search now, thanks for your answer. – John May 09 '18 at 17:20

1 Answers1

2

You could use the copyWebpackPlugin: https://webpack.js.org/plugins/copy-webpack-plugin/

It basically copy your file or folder from a to b

in webpack.config:

const CopyPlugin = require("copy-webpack-plugin");
...
plugins: [
    new CopyPlugin([
        {
            from: path.resolve(__dirname, "src/pathTo/giantJsonFolder"),
            to: "assets",
        },
    ]),
],

Then fetch it where you'll need it:

const response = await fetch('assets/filename.json')
const json = await response.json();
console.log(json)