这是indexloc提供的服务,不要输入任何密码
Skip to content

Conversation

@rsamoilov
Copy link

Storage#write_file is not thread-safe.

The bug comes out when multiple threads are trying to write data into one index file.
When such situation occurs, one thread moves tmp_file to new_file by FileUtils.mv. At the same time second thread tries to move temporary file to new_file too, but temporary file not exists already, so method fails with Errno::ENOENT: No such file or directory.

You can reproduce it with concurrent creating indexed model instances, something like this:

threads = []
5.times do
  5.times do

    threads << Thread.new { IndexedModel.create! }

  end
end

threads.each &:join

One possible fix is to create temporary file with uniq name. It is possible to create it via Tempfile class or manually, as in pull request.
With this fix, FileUtils.mv will move only temporary file from the current thread.

@dougal
Copy link
Owner

dougal commented Sep 4, 2012

Thanks for the bug report.

Unfortunately this patch introduces another bug, whereby the data is written but is inconsistent.

What OS, ruby version etc are you using?

@rsamoilov
Copy link
Author

I am using Ubuntu 12.04 and ruby 1.9.3.

@rsamoilov
Copy link
Author

Yes, I see the issue. I think, the problem is with Marshal.load and Marshal.dump invocations in Storage#operate. Data is being loaded while new data is not dumped yet.

I'll look for solution and update pull request.

@dougal
Copy link
Owner

dougal commented Sep 5, 2012

The solution is to add a mutex lock on the relevant file.

There is already a process lock in place, but no thread lock.

@dougal
Copy link
Owner

dougal commented Nov 25, 2012

Merged in development branch, thanks Roman, a great contribution.

@dougal dougal closed this Nov 25, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants