2

I am building a functionality which allows clients to crawl their website, get the links and download the html files. Word count is done on the downloaded files and the cost estimate is shown for translation of the webpages.

I want to make sure that only one estimation is running at a time. So if there are two clients trying to get estimate for their website at the same time, only one clients estimation process will run and other will have to wait.

For this I tried using RabbitMQ which was suggested to me. I am using Coldfusion with framework/1. I used the Java examples given in rabbitmq website. I can send messages but I am having trouble consuming messages. The following function is creating a object and at the same time overriding a function on the fly. This is not possible in coldfusion. I tried getting help in slack and google groups but I didnt get a definitive answer.

final Consumer consumer = new DefaultConsumer(channel) {
  @Override
  public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
    String message = new String(body, "UTF-8");

    System.out.println(" [x] Received '" + message + "'");
    try {
      doWork(message);
    } finally {
      System.out.println(" [x] Done");
    }
  }
};
channel.basicConsume(TASK_QUEUE_NAME, true, consumer);

I am not well acquainted with Java. I tried my best to convert the above steps in coldfusion. I didnt succeed. I tried the following and I dont even know if it is the right way to do it.

consumer = javaloader.create("com.rabbitmq.client.DefaultConsumer").init(channel);
envelope = createObject("java","com.rabbitmq.client.Envelope");
properties = createObject("java","com.rabbitmq.client.BasicProperties");
consumerTag = toString(consumer.getConsumerTag());
body = javaCast("byte[]",[]);
            consumer.handleDelivery(consumerTag,envelope,properties,body);

The error I got when I run the above code

I have wasted way more time than I need to on this. I need two way communication. I want the estimation process to keep informing the user on every step like crawl started, crawl finished and so on.

I just want to know if it is worth trying to make rabbitmq work or is it even the right tool for the job? I can simply create a table in the database and have a flag in it to queue the process. Which is the best way to achieve the functionality? Are there other useful tools for queueing tasks in coldfusion?

P.S. I have also tried the following solution: https://github.com/lmajano/messaging-polyglot

  • *This is not possible in coldfusion* Well technically it is possible, but only with [interfaces](http://docs.oracle.com/javase/tutorial/java/concepts/interface.html), CF10+. Unfortunately DefaultConsumer is a concrete class, so in this case you are correct. That specific code cannot be easily simulated in CF. *I didnt succeed* How so? You say neither one worked, but do not really explain *how* . Are you getting an error, is nothing happening (The API says handleDelivery is a no-op). Same question about the other project. – Leigh Feb 02 '16 at 21:46
  • @Leigh I apologise for not being through enough. I have added the error message screenshot above. I have very limited knowledge of java so it was hard understanding most the code. In the google group for rabbitmq I was suggested to "Simply subclass DefaultConsumer or implement Consumer from scratch." I got even more confused with the answer. –  Feb 02 '16 at 22:13
  • Yes. That makes complete sense in java. It is like overriding a parent component function in a CFC. Unfortunately, when the thing you are overriding *from* CF is a java class method, it is more involved - and not possible at all with concrete classes. Regarding the error, at first glance it looks correct, ie number of params, types, order. So without running it, I am not sure why it is complaining. What about the other project? Looks like it has a cfml example. That said, confirming approach is a probably good place to start ie as No point pursuing the lib if it is not the right tool. – Leigh Feb 02 '16 at 22:46
  • I am assuming that I am making mistake while declaring variables and instantiating objects to be passed into the handleDelivery function. I can just run the java using shell scripts and cfexecute but I wanted to do it inside coldfusion. The possibilities seem bleak. The cfml example is useful. It uses Queueing Consumer class instead of default consumer and also uses dynamic proxy, threads. I have limited knowledge of them so I was hesitant to use them. I think I will go with database implementation. I really wanted to make this work. I will try it in free time, for now I cant. –  Feb 02 '16 at 23:33
  • 1
    I hear you, but .. I just noticed something off about the code. It mixes `javaloader.create` and `createObject` calls, which could be the problem. (The full stack trace would have more details.) You typically use one or the other, not both. Reason being classloaders are notoriously picky and territorial and [do not always play well with others](http://stackoverflow.com/questions/15506237/getting-coldfusion-called-web-service-to-work-with-javaloader-loaded-objects/15592095#15592095). Change the code so it either uses `javaloader.create` -or- `createObject` for ALL for all of the object calls. – Leigh Feb 03 '16 at 00:27
  • Thank you for the concern Leigh. I used Javaloader at the start. When I ran it, it gave me `No matching Method/Function for com.rabbitmq.client.DefaultConsumer.handleDelivery(string, coldfusion.runtime.java.JavaProxy, coldfusion.runtime.java.JavaProxy, binary) found`. I am running railo server so I thought of trying railo object. Nothing worked. I can use QueueingConsumer class and use nextDelivery function to access the message. But nextDelivery goes slow and catches one message from multiple messages. Anyways, I have given up for now. I will inform you if I make any progress in the weekend. –  Feb 03 '16 at 01:03
  • Well for [Railo's createObject](http://blog.getrailo.com/post.cfm/railo-tip-createobject-java), you need to specify the jar path. Unless you put the jar files in the "class path" and restarted the server. Anyway, it sounds like we are both busy, so we will see what the weekend brings. – Leigh Feb 03 '16 at 02:10
  • Yes I have done that. I specified the classpaths in the java settings in application.cfc. Without that the createObject wouldnt have worked and produced an error. Thanks for your time even when you are busy. –  Feb 03 '16 at 03:04

1 Answers1

1

I don't know that there is a single best way to achieve the result you wish to have. One thing is certain: RabbitMQ alone is not the solution to the problem.

Rabbit is designed as a high-speed message broker. It does not worry about the content of messages - it is more concerned with routing and delivery. The problem you have described is one of message processing. In fact, there are two problems:

  1. How to ensure that only one crawl task operates for a given web site
  2. How to give status feedback to the user

To solve the first issue, you would need some type of task pre-processor, which would check for running tasks prior to enquing a duplicate crawl task. You could utilize a database or some other type of distributed cache (e.g. memcached).

The second issue is a bit more complicated and there are many ways you could achieve it. Most likely, you would publish events when the status of a running task is updated, which would then reach the user.

Neither solution really has anything to do with how RabbitMQ works or what its role is within your system.

theMayer
  • 15,456
  • 7
  • 58
  • 90
  • For the first problem I have decided to use database and set a flag for isRunning. For the second problem I will just use coldfusion to notify user on each step started and completed. I was trying to setup rabbitmq to make work queues so that I dont have to query database to check on every step. –  Feb 02 '16 at 22:19