SATA errors on Linux with Samsung SSD 840 Series with Asus M2N-E


This problem had been driving me crazy for weeks.  I run a home linux server (currently Fedora Core 18) as a NAS and various other services.   Ever since I upgraded the OS disks to SSD I began getting SATA errors in my kernel logs:

Jul 31 00:02:26 kernel: [20859.208310] ata3:EH in SWNCQ mode,QC:qc_active 0x7E sactive 0x7E
Jul 31 00:02:26 kernel: [20859.208371] ata3: SWNCQ:qc_active 0x3E defer_bits 0x40 last_issue_tag 0x5
Jul 31 00:02:26 kernel: [20859.208482] ata3: ATA_REG 0x41 ERR_REG 0x84
Jul 31 00:02:26 kernel: [20859.208533] ata3: tag : dhfis dmafis sdbfis sactive
Jul 31 00:02:26 kernel: [20859.208585] ata3: tag 0x1: 1 0 0 1
Jul 31 00:02:26 kernel: [20859.208636] ata3: tag 0x2: 1 0 0 1
Jul 31 00:02:26 kernel: [20859.208697] ata3: tag 0x3: 1 0 0 1
Jul 31 00:02:26 kernel: [20859.208748] ata3: tag 0x4: 1 0 0 1
Jul 31 00:02:26 kernel: [20859.208800] ata3: tag 0x5: 0 0 0 1
Jul 31 00:02:26 kernel: [20859.208860] ata3.00: exception Emask 0x1 SAct 0x7e SErr 0x0 action 0x6 frozen
Jul 31 00:02:26 kernel: [20859.208914] ata3.00: Ata error. fis:0x21
Jul 31 00:02:26 kernel: [20859.208967] ata3.00: failed command: READ FPDMA QUEUED
Jul 31 00:02:26 kernel: [20859.209049] ata3.00: cmd 60/08:08:e8:23:bb/00:00:04:00:00/40 tag 1 ncq 4096 in
Jul 31 00:02:26 kernel: [20859.209251] ata3.00: status: { DRDY ERR }
Jul 31 00:02:26 kernel: [20859.209303] ata3.00: error: { ICRC ABRT }
Jul 31 00:02:26 kernel: [20859.209354] ata3.00: failed command: READ FPDMA QUEUED
Jul 31 00:02:26 kernel: [20859.209410] ata3.00: cmd 60/08:10:d8:26:bb/00:00:04:00:00/40 tag 2 ncq 4096 in
Jul 31 00:02:26 kernel: [20859.209605] ata3.00: status: { DRDY ERR }
Jul 31 00:02:26 kernel: [20859.209675] ata3.00: error: { ICRC ABRT }
Jul 31 00:02:26 kernel: [20859.209734] ata3.00: failed command: READ FPDMA QUEUED

Since the spinning SATA disks I replaced also gave me errors (I assumed they were dying...), it was a bit of a mystery.  Is the SATA controller dying?  The errors did seem to correlate with greater disk utilization.

I swapped the cables.  I swapped the port.  For a day I convinced myself the problem was associated with the port.  I forced the kernel to throttle the SATA to 1.5Gb/s (libata.force=1.5G).  I tried libata.noacpi=1.

Finally I found the answer: libata.noncq

It's even a bit obvious now in the log messages: SWNCQ

Apparently newer drives let the kernel offload the write ordering to the drives.  After all, the drive knows its physical properties better than the kernel:  http://en.wikipedia.org/wiki/NCQ
I did file a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1013229

Now that I've added libata.nonncq to the kernel command line, my errors have gone away.  I'll try removing it someday when I upgrade the motherboard or OS.  I suspect it is the sata controller on the motherboard.



Comments

Post a Comment

Popular posts from this blog

Repairing a Moenstone Composite Granite Sink

A few years ago we remodeled our kitchen and went with an undermount Moenstone sink. This Thanksgiving I noticed a hairline crack in the bottom. We called Moen and they were willing to give us money or a new sink as part of the warranty (which is great!), but the sink is undermounted. We'd probably need to remove the entire countertop to put a new one in, and Moen no longer makes them. Our choices were either to repair it, install a drop in sink, or maybe find another undermount sink with a similar profile. A new undermount probaby requires taking off the countertop and recutting.  A drop-in requires recutting the counter.  Ugh! So... I researched how to repair composite granite sinks.  I didn't find a whole lot of information. Some people suggested epoxy. We are one day (2015-12-19)  into our repair, still waiting for the glue to set, but I wanted to document my process. The foundation for our repair is a Simpson Strong-Tie product I found for repairing c...
Read more

When to ask for help when stuck on a technical problem

I thought this blog post by  +Matthew Ringel was a concise and useful summary of how (and when) to ask for help when stuck on a technical problem: https://blogs.akamai.com/2013/10/you-must-try-and-then-you-must-ask.html I often find coworkers skipping step #1: working at the problem a little longer and documenting/recording/reviewing what you've already tried. When I do step #1 I often solve the problem myself.  I might be in the middle of an email explaining what the problem is.  I owe it to them to explain what I've tried, what results I got, etc.  Most of the time the solution then presents itself!  I then just delete the email draft and keep plugging away. But then sometimes I wait too long for step #3 - going and asking for help. And then sometimes, I just need a rubber duck: http://www.rubberduckdebugging.com/
Read more

Western Digital Green Drives and Linux - They're dying!

The last time I swapped the drives out in my home built NAS, I bought Western Digital Green drives.  They sounded great!  Low power, low heat, low noise... Unfortunately, they appear to not have been built for this particular application.  They auto park and the timer is set too low for Linux filesystems.  (I believe the default may be 8 seconds! http://techreport.com/forums/viewtopic.php?f=5&t=78891) My drives began to give me trouble (thankfully) even before the warranty ran out.  I found the load cycle count on my drive to be extremely high - 10's or 100's of thousands.  This is way beyond the expected count. Here's how to query for it: # smartctl -A /dev/sdX | grep ^193 193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       XXXXXXX Luckily there is a solution -...
Read more