How to mine text form an openEdX forum?

Question

I would like to collect all the message on an openEdX forum for quantitative analysis. What are the most frequent specialized words uses, etc. But how to download / mine an openEdX forum ?

Answer 1

I suppose you have access to the Open edX platform? you need to extract text from all threads and all messages. The library for interacting with the forums is in lms.lib.comment_client. For example, to obtain all threads from a specific course, execute:

lms.lib.comment_client.Thread.search({
    'course_id': course_id,
})

For each thread, you must then iterate over the comments and extract their body, and eventually their title. I believe you can obtain the comments from a thread using the thread.to_dict() method.