1

I have a small ruby program that logs into an sftp server, pulls a couple filenames from a directory on the server, and then writes those filenames to a sqlite3 database.

I'm saving the filenames from the sftp to an array. I can confirm the array is created and has values, however when I try to use data from that array to save to a database I get an error, and honestly I can't figure out what the error means? or why the data from the array doesn't get written?

The other strange thing is I can manually create an array with the exact same values as came from the sftp, and use the same data from that new array to successfully save to the db. Why does an array created from sftp data cause an error, but an array I manually create with the exact same data, successfully write to a db?

Thanks for your help!

My Program (file: xb.rb):

# db
require 'active_record'
ActiveRecord::Base.establish_connection(
  :adapter => 'sqlite3',
  :database => 'D:/prog/test/db.sqlite3.db'
)
class Db < ActiveRecord::Base
end

# connect to sftp and get last 2 filenames, put them in array
require 'net/sftp'
@files = []
Net::SFTP.start('XXX.XXX.XXX.XXX','xxx',:password => 'xxx') do |sftp|
    sftp.dir.foreach("/Outgoing/") do |sftp_file|
    @files.push(sftp_file.name)
  end
end

# Get last 2 filenames
@files = @files.sort!.pop(2)
# Show array to confirm it has values and that it's an array
puts @files
puts @files.class

# Write filenames to db
@files.each do |new_file|
  Db.create(:name => new_file)
end

Returned:

PS D:\prog\test> ruby xb.rb
20120823002349.txt
20120824012407.txt
Array
D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/sqlite_adapter.rb:212:in `type_cast': NoMethodError: undefined method `error' for nil:NilClass: INSERT INTO "dbs" ("name") VALUES (?) (ActiveRecord::StatementInvalid)
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/sqlite_adapter.rb:264:in `block (2 levels) in exec_query'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/sqlite_adapter.rb:263:in `map'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/sqlite_adapter.rb:263:in `block in exec_query'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:280:in `block in log'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activesupport-3.2.1/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:275:in `log'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/sqlite_adapter.rb:247:in `exec_query'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:61:in `exec_insert'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:88:in `insert'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:14:in `insert'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/relation.rb:66:in `insert'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/persistence.rb:363:in `create'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/timestamp.rb:57:in `create'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/callbacks.rb:268:in `block in create'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activesupport-3.2.1/lib/active_support/callbacks.rb:403:in `_run__619764720__create__513830955__callbacks'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activesupport-3.2.1/lib/active_support/callbacks.rb:405:in `__run_callback'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activesupport-3.2.1/lib/active_support/callbacks.rb:385:in `_run_create_callbacks'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activesupport-3.2.1/lib/active_support/callbacks.rb:81:in `run_callbacks'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/callbacks.rb:268:in `create'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/persistence.rb:344:in `create_or_update'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/callbacks.rb:264:in `block in create_or_update'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activesupport-3.2.1/lib/active_support/callbacks.rb:414:in `_run__619764720__save__513830955__callbacks'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activesupport-3.2.1/lib/active_support/callbacks.rb:405:in `__run_callback'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activesupport-3.2.1/lib/active_support/callbacks.rb:385:in `_run_save_callbacks'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activesupport-3.2.1/lib/active_support/callbacks.rb:81:in `run_callbacks'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/callbacks.rb:264:in `create_or_update'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/persistence.rb:84:in `save'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/validations.rb:50:in `save'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/attribute_methods/dirty.rb:22:in `save'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/transactions.rb:241:in `block (2 levels) in save'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/transactions.rb:295:in `block in with_transaction_returning_status'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:190:in `transaction'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/transactions.rb:208:in `transaction'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/transactions.rb:293:in `with_transaction_returning_status'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/transactions.rb:241:in `block in save'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/transactions.rb:252:in `rollback_active_record_state!'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/transactions.rb:240:in `save'
        from D:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/activerecord-3.2.1/lib/active_record/persistence.rb:45:in `create'
        from xb.rb:27:in `block in <main>'
        from xb.rb:26:in `each'
        from xb.rb:26:in `<main>'

But then if I manually create an array (@files2) with the exact same values as @files (I compare @files2 against @files to confirm), I can successfully use the data from @files2 to write to the db.

Test (appended to previous program):

.
.
.
# create manual array with same data as @files
@files2 = ["20120823002349.txt","20120824012407.txt"]
puts @files == @files2

# Write filenames to db
@files2.each do |new_file|
  Db.create(:name => new_file)
end

Returns:

PS D:\prog\test> ruby xb.rb
20120823002349.txt
20120824012407.txt
Array
true

The test wrote the array values to the db. I'm perplexed, why can't I use the data from the sftp array (@files) to write to the db, when @files2 has the exact same data and can be used to write to db?

I appreciate any help, thoughts, ideas.

ecbp
  • 11
  • 1
  • Are you sure `@files` contains what you think it does? What does `puts @files.inspect` have to say? And does the `Db` class do anything interesting? – mu is too short Aug 24 '12 at 19:01
  • Thanks mu, good point; `puts @files.inspect => ["20120823002349.txt", "20120824012407.txt"]`. Which is what I expected, but that confirms it. The `Db` class doesn't do anything except inherit from ActiveRecord which let's me make the `Db.create()` call later to write to the db. – ecbp Aug 24 '12 at 21:17
  • You do have a `dbs` table in that SQLite file, right? I feel like I must be missing something obvious. – mu is too short Aug 25 '12 at 03:33
  • well i created a dbs table with name attribute in a new sqlite3 database and ran your code against it. It worked absolutely fine. My platform -database and ruby code on windows and ftp server - linux. Perhaps you're getting something unexpected returned from your ftp server. Try running same code on some directory other than /Outgoing/ – saihgala Aug 25 '12 at 07:12
  • @muistooshort, yup I have the table and SQLite db set up, I can write values from `@files2` successfully, just not values from `@files`. – ecbp Aug 25 '12 at 17:12
  • Thanks @ashish, that's good news, thanks for checking it out. Just to confirm, did you use an FTP or sftp server? I ran ruby code on windows, I don't control the sftp server so not sure what platform. I'll try another folder on the sftp and try running from my Mac at home using an older ruby. I was thinking maybe the sftp part isn't blocking and so I would be trying to write to the db before `@files` was being properly defined? Not sure. – ecbp Aug 25 '12 at 17:23
  • i used ftp, basically word to word same code that you have posted in the question and it works like a charm on Windows and using ruby 1.9.2. So it could be either a ruby version issue, i see you used 1.9.3 (but it should be unlikely) or more likely as I said before there is some output your ftp server is sending back which doesn't go down well with your code. – saihgala Aug 25 '12 at 18:12
  • Looks like it was something with ActiveRecord. Updated gem to 3.2.8 and everything is well. Lesson learned. I appreciate your time on this! – ecbp Aug 27 '12 at 20:59

0 Answers0