-1

We have log files of mySQL database and we want to use that log files for analysis (data mining, machine learning,...) and iam very new with that.

Can you give me instruction how to do that ?

Has QUIT--Anony-Mousse
  • 76,138
  • 12
  • 138
  • 194
user2863110
  • 21
  • 2
  • 7
  • I think you need to more clear in terms of what you intend to do with log file. Most actions can simply be achieved using bash commands like grep etc. – MSameer May 05 '16 at 05:13

1 Answers1

0

There are a number of tools that could be useful to you depending on your requirements.

But why use the MySQL logs and not the DB directly, or have the details you are searching for go to a new 'information mining' table in the DB as the user interacts with the interface. Which could be a lot more powerful

If you wish to determine the rate of inserts / queries. You can actually set up MySQL to log these types of events to different files (so some of the work is already done, rather than having to mine a complete log file of all events). Otherwise you are going to want to make use of tools such as Grep.

On top of that you have a system called Lucerne (from apache) that will mine the data and search for key words. They have various different hooks for going into Java, C, and others. Its very similar to how google trawl web pages.

Otherwise if you intend to mine the data 'within' the database, then the logs are clearly not your best point of call.

The logs will contain lots of information about the users also. IP address may be more tricky, but you could cross reference the name of the user running the query and the general server logs to determine the IP of the connection.

DaveM
  • 734
  • 4
  • 11
  • 35
  • Thanks DaveM, I use Python and now i want to access (parsing) log files of mySQL database for retrieve data. Can you give me instruction ? – user2863110 May 05 '16 at 08:40
  • @user2863110 are you needing to access information that is stored in the DB? if so you don't need to access the log files, you just need to send the query to the DB directly (in SQL). Otherwise what sort of detail are you hoping to extract from the logs ? Add more info into your original question to clarify. – DaveM May 09 '16 at 10:50
  • @DaceM : http://stackoverflow.com/questions/37176398/owa-open-web-analytics-data-access-api-with-python Please read this question and give me what do you think. – user2863110 May 12 '16 at 02:49
  • @user2863110 : dudu, must appologise, i've never done that sort of work either. I would propose you try to make a solution in python, then when you get an error message, place that and your code into your question. – DaveM May 12 '16 at 07:59