Docker For Mac Mysql Slow

ferafheat.netlify.com › Docker For Mac Mysql Slow ∎∎∎

Fixes docker/for-mac#2216, docker/for-mac#2375 Fix filesystem event delivery to Kubernetes pods when the path to the bind mount is a symlink. Fix a race between calling stat on a file and calling close of a file descriptor referencing the file that could result in the stat failing with EBADF. For certain reason it's 15 times slower than Virtual Box on the same machine with. Is there anyway we can optimize MySQL performance in docker? Also happens on Docker for Mac mysql imports take a very long time.

Docker just released a native MacOS runtime environment to run containers on Macs with ease. They fixed many issues, but the bitter truth is they missed something important. The read and write access for mounted volumes is terrible.

Benchmarks

We can spin up a container and write to a mounted volume by executing the following commands:

SafariCacheExplorer is the unique tool for explore the Safari's cache (and any other native Mac OS X cache). With SafariCacheExplorer you can retrieve an image, video, sound, style sheet from your. SafariCacheExplorer 2.0 for Mac is available as a free download on our application library. The most popular versions of SafariCacheExplorer for Mac are 1.4, 1.3 and 1.2. The unique ID for this app's bundle is com.beecubu.SafariCacheExplorer. Safaricacheexplorer 2.0 free download for macr mac os x.

Start a container
Mount the current directory
Write random data to a file in this directory

Let’s compare the results of Windows, Cent OS and Mac OS:

Windows 10

Cent OS

Mac OS

So the winner is…. 19 seconds for writing. For reading it is quiet similar. When you develop a big dockerized application then you are in a bad spot. Usually you would work on your source code and expect no slowdowns for building. But the bitter truth is it will take ages.

This GitHub issue tracks the current state. There is a lot of hate so better listen to the “members” instead of reading all the frustrations.

@dsheetz from the Docker for Mac team nailed the issue:

Perhaps the most important thing to understand is that shared file system performance is multi-dimensional. This means that, depending on your workload, you may experience exceptional, adequate, or poor performance with osxfs, the file system server in Docker for Mac. File system APIs are very wide (20-40 message types) with many intricate semantics involving on-disk state, in-memory cache state, and concurrent access by multiple processes. Additionally, osxfs integrates a mapping between OS X's FSEvents API and Linux's inotify API which is implemented inside of the file system itself complicating matters further (cache behavior in particular).

At the highest level, there are two dimensions to file system performance: throughput (read/write IO) and latency (roundtrip time). In a traditional file system on a modern SSD, applications can generally expect throughput of a few GB/s. With large sequential IO operations, osxfs can achieve throughput of around 250 MB/s which, while not native speed, will not be the bottleneck for most applications which perform acceptably on HDDs.

Latency is the time it takes for a file system system call to complete. For instance, the time between a thread issuing write in a container and resuming with the number of bytes written. With a classical block-based file system, this latency is typically under 10μs (microseconds). With osxfs, latency is presently around 200μs for most operations or 20x slower. For workloads which demand many sequential roundtrips, this results in significant observable slow down. To reduce the latency, we need to shorten the data path from a Linux system call to OS X and back again. This requires tuning each component in the data path in turn -- some of which require significant engineering effort. Even if we achieve a huge latency reduction of 100μs/roundtrip, we will still 'only' see a doubling of performance. This is typical of performance engineering, which requires significant effort to analyze slowdowns and develop optimized components.

Many people created workarounds with different approaches. Some of them use nfs, Docker in Docker, Unison 2 way sync or rsync. I tried some solutions but non of them worked for my docker container that contains a big Java monolith. Either they install extra tools like vagrant to reduce the pain. Vagrant uses nfs but this is still slow compared to native write and read performance. Or they are unreliable, hard to setup and hard to maintain.

I made a step back and thought about the root issue again. A very good approach is docker-sync. It’s a ruby application with a lot of options. One very mature option is file synchronisation based upon rsync.

Rsync

Rsync initial release was in 1996 (20 years ago). It’s used for transferring files across computer systems. One important use case is 1-way synchronization.

Sounds good…, right ?

Docker-sync supports rsync for synchronization. In the beginning it worked but a few days later I got some connection issues between my host and my container.

Do you know the feeling when you want to fix something but it feels so far away? You realise you don’t understand what’s happing behind the scenes.

The rsync approach sounds right. It tackles the root of the issue: operating on mounted files right now is damn slow.

I tried other solutions but without real success.

Build a custom image

So let’s try to get our hands dirty. You start a rsync server in the container and connect to it using rsync. This approach works for many years for other use-cases.

Let’s setup a docker Centos 6 container with an installed and configured rsync service.

The Dockerfile

2. Build the container within the repository directory.

3. Start the container and map the rsync server port to a specific host port.

Now we need to sync our share directory and sync any changes again as soon as anything changes. Rsync will only sync changes after an initial sync.

Fswatch utilizes rsync to talk to the container as soon as something changes. We do not use any kind of docker volume mounting. Hence all file operations will stay in the container and will be fast. Whenever we change something rsync transfers it to the container using . For sure you can use all rsync features like delete rules or exclude patterns.

If we change something (it does not matter if it’s a small project or a huge one) then we see something like

0.02 seconds, great !

Fswatch uses file system events on Mac OS. Thus is still very fast and you can event tweak it. For example by excluding build related directories like target or node_modules.

Sources are available on GitHub.

Mac OS X finally gets ZFS support thanks to former Apple engineer. Apple has long been rumored to include support for the ZFS file system designed by Sun, with rumors dating back even before Sun CEO Jonathan Schwartz announced in 2007 Leopard would utilize ZFS as its file system. Zfs has finally arrived for mac. Apple Clarifies Status of ZFS File System In Mac OS Apple has included a read-only version ZFS in Leopard as a future file system option for high-end storage systems. Apple now says that the ZFS file system will be available in the forthcoming Mac operating system, sort of. Servers typically have ample resources, like RAM, to supply large amounts of file system bandwidth. The typical Mac desktop, on the other hand, has limited resources. So our ZFS implementation has been adapted to function where kernel resources are constrained. We have, in essence, put ZFS on a resource diet. Our custom ZFS memory manager is.

For small projects the bad performance is not a critical issue. For huge application rsync is our hero. Good old tools, and still reliable and important.

Especially for all guys who love Mac OS and need to use a VM know the pain. Issues like the command key mapping are annoying. Either you map it to the Windows key or in the end you don’t use it anymore. So on Mac OS you use cmd+c to copy something and in your container you use control. For sure you can also map your host control to command but then you have again other issues. Everything is better when you can work in Mac OS instead of in a virtual machine as a mac user.

I hope you enjoyed the reading. If you like it and feel a round of applause, drop a comment and subscribe our mailing list.

I work with ubuntu-technology. We are a very young software engineering company located in Dresden, Germany. We are looking for long term projects and partners. If you need support in certain software engineering or AI-related fields, drop a message in our contact form.

Happy coding :)