I am creating an API. Demo code is on Pull-data-demo-code.
Related bean classes:
import com.alibaba.fastjson.JSONObject;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
@Data
@AllArgsConstructor
@Builder
@NoArgsConstructor
public class Response {
private Integer code;
private String message;
private JSONObject info;
public static Response success(JSONObject data) {
return Response.builder().code(200).message("success").info(data).build();
}
public static Response error(JSONObject data) {
return Response.builder().code(500).message("fail").info(data).build();
}
}
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.util.List;
@Data
@AllArgsConstructor
@NoArgsConstructor
public class ResultBean {
private Long id;
private Long parentId;
private String publishTime;
private String title;
private String content;
private List<ResultBean> children;
}
DataController is like:
import com.example.demo.entity.Response;
import com.example.demo.service.DataService;
import org.springframework.web.bind.annotation.CrossOrigin;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import javax.annotation.Resource;
@RestController
@CrossOrigin
public class DataController {
@Resource
private DataService dataService;
@PostMapping(value = "/api/v1/data/analyse")
public Response pullDataController(@RequestParam(value = "keyword") String keyword) {
return Response.success(dataService.analyseData(keyword));
}
}
DataService is like:
import com.alibaba.fastjson.JSONObject;
public interface DataService {
/**
* Analyse data with keyword submitted from client.
*
* @param keyword keyword
* @return the analysed result as json
*/
JSONObject analyseData(String keyword);
}
DataServieImpl is like:
import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.example.demo.entity.ResultBean;
import com.example.demo.service.DataService;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.IOUtils;
import org.springframework.stereotype.Service;
import java.io.IOException;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
@Service
@Slf4j
public class DataServiceImpl implements DataService {
/**
* Mock third-party api path.
*/
private static final String THIRD_PARTY_API_PATH = "http://***.com/api/v1/data";
@Override
public JSONObject analyseData(String keyword) {
JSONObject result = new JSONObject();
// TODO pull data and save to database for at most 5 seconds.
pullData(keyword);
// analyse data
List<ResultBean> resultBeanList = fetchDataAndAnalyse(keyword);
result.put("data", resultBeanList);
return result;
}
private void pullData(String keyword) {
List<ResultBean> resultBeanList = new ArrayList<>();
String response;
try {
response = IOUtils.toString(new URL(THIRD_PARTY_API_PATH + keyword), StandardCharsets.UTF_8);
JSONObject json = JSON.parseObject(response);
if (json != null) {
Integer num = json.getInteger("childrenNum");
String subKeyword = json.getString("subKeyword");
// raw data, can be converted to class and save to database.
JSONArray data = json.getJSONArray("data");
saveRawDataToDatabase(data);
if (num != null && num > 0) {
// Iterate for more data
pullData(subKeyword);
}
}
} catch (IOException e) {
log.error("error message:{}", e.getMessage());
log.error("error detail:{}", e.toString());
}
}
private void saveRawDataToDatabase(JSONArray data) {
if(data != null && data.size() > 0) {
for (int i = 0; i < data.size(); i++) {
JSONObject jsonObject = data.getJSONObject(i);
// ResultBean resultBean = jsonObject.toJavaObject(ResultBean.class);
// dataDao.insert(resultBean);
}
}
}
private List<ResultBean> fetchDataAndAnalyse(String keyword) {
List<ResultBean> list = new ArrayList<>(100);
// Read data from database
// List<ResultBean> list = dataDao.query(keyword);
// process list, handle parent-children relation and return result.
return list;
}
}
As you can see in DataServiceImpl,
pullData(String keyword)
is a recursive function, so it can be quite time-consuming. How to restrict the running time of this function to at most 5 seconds?