Bug 105871

Created attachment 46703 [details]
udev log

udev log of the usb attach

Created attachment 46704 [details]
hald.log

hald log file

This could be the same issue as bug 105798 

Can you attach /proc/bus/usb/devices with the device plugged in?

It sounds like you do not have the usb2 host controller driver loaded (ehci-hcd)
and the device is running at usb 1 rates, but need to see that file to make sure
or not.

Created attachment 46721 [details]
/proc/bus/usb/devices output

Looks like it is loaded...

horas:~ # lsmod | grep hci
uhci_hcd	       36384  0
ohci_hcd	       22916  0
ehci_hcd	       36744  0
usbcore 	      140852  7
usb_storage,usbserial,usbhid,uhci_hcd,ohci_hcd,ehci_hcd

Yeah, it's connected, thanks.

Are there _any_ kernel log messages that show up when you plug the device in
and start talking to it?

There are no kernel messages during random access.

please provide the full hwinfo output. 

Created attachment 46849 [details]
full hwinfo log

Here is the full hwinfo log

According to the logs and other info, the device is operating correctly at
480 Mbps (high-speed). It's likely a limitation of the device's flash
memory that it can't handle a faster write rate.

It works perfectly under SUSE 9.3 and Windows

Thanks for that datapoint.

Another possible reason for a speed regression from 9.3 to 10.0 is that
in recent kernels, sync write support was added to the vfat filesystem.

That could slow down writes considerably and is controlled by a mount option. 

You can verify that is your case by running:

cat /proc/mounts | grep vfat

If 'sync' is one of the options listed, then it's the case.

can you post the /proc/mounts output?
Is it also slow if mounted manually?  
mount /dev/sdaN /mnt 
dd if=/dev/zero of=/mnt/foo bs=1M count=1

/dev/sda1 on /media/usbdisk type subfs
(rw,nosuid,nodev,sync,fs=floppyfss,procuid,utf8=true)

I've mounted the device without the sync parameter and hey it's fast as hell :)

 find * -type f -print0 | env -i xargs -0 grep -w MS_SYNCHRONOUS 

this finds nothing but nfs, no filesystem forces sync. So it must come from subfs.

grep -r HOTPLUG_MOUNT_SYNC /etc/sysconfig , set it to no or off or whatever.
submount seems to look for some hal flags. Its not a kernel bug.

HOTPLUG_MOUNT_SYNC is only valid for HOTPLUG_MOUNT_TYPE=fstab, not for subfs.

Just for information: previously (SuSE 9.3) it took around 10 seconds to copy  
my history.dat to my USB-2.0 memory stick; with beta3 it takes now around 120  
seconds. 

Klaus, did you check if the module ehci_hcd was loaded orderly on your box?

I found a documentation which describes how to disable the sync mount options of
a subfs mounted device.

http://portal.suse.de/sdb/de/2005/06/dkukawka_halnosync_html.html
https://bugzilla.novell.com/show_bug.cgi?id=85413#c5

Mounting hotplugged devices without "sync"
-------------------------------------------

If writing to USB storage devices like USB sticks or USB hard drive is
rather slow or if you do not want synchronized writing to these devices,
disable the sync mount option.

1.
Create the directory /usr/share/hal/fdi/policy/95userpolicy:

mkdir -p /usr/share/hal/fdi/policy/95userpolicy/

2.
Create a file with the name nosync.fdi and the following contents:

vi /usr/share/hal/fdi/policy/95userpolicy/nosync.fdi

<?xml version="1.0" encoding="UTF-8"?>
<deviceinfo version="0.2">   
  <device>   
    <!-- disable sync for mount -->   
    <match key="block.is_volume" bool="true">   
      <match key="volume.fsusage" string="filesystem">   
        <match key="volume.uuid" string="==UUID==">   
          <merge key="volume.policy.mount_option.sync" type="bool">false</merge>   
        </match>   
      </match>   
    </match>   
  </device>   
</deviceinfo>

3.
Adjust the line <match key="volume.uuid" string="==UUID=="> according to
your hardware. Run 'lshal' to retrieve the "volume.uuid" and enter this id
instead of ==UUID==

4.
Finally restart the HAL service with

rchal restart

I noticed the same problem with my device. 
 
I tried the manual mount without sync and that seems to do the trick: 
 
- Copying file with normal HAL mount (1.5 MB): 1:06 (plus a few seconds w/ 
light still blinking on device) 
- Copying file with manual mount (1.5 MB): 0:04 (includes copy and umount) 
 
Obviously this default behavior is unacceptable, but we can't mount without 
synchronization either or people will pull out their devices and loose their 
data.  I don't know what 9.3 did, but it seemed to combine speed with data 
synchronization quite well (I never manually umounted my devices and I never 
lost data).    

These might be cause by the wrong USB speed detection. Olaf ? 

no, kernel does everything alright, because manual mounting gives high performance.

either do not mount with sync or trick mason and axboe into fixing the kernel,
by providing a sync_interval mount option or some other blocklayer tweaks to
sync every N seconds.

we discussed the sync_on_close mode, which would solve this issue in every 
case, but this seems to not to be possible according to some irc discussion, 
is that right Chris / Jens ?  

Definitely possible, but not 1 week before release.  This can actually be coded without  
changing specific filesystems and the patch will be rather small.  
  
I don't think including it right now is  a great idea, you'll have to convince AJ first. 

sure, not for 10.0 ... I just got a negative answer some month ago. 
 
However, can we agree that we should do this for 10.1 and close this bug with 
LATER now ? 
 
AJ, can you add it to the feature document or shall I ? 

Adrian, could you do it, please?

Moving on to 10.1

removing sync seems to workaround this issue, but since we do mount with sync 
since quite some time and Andreas said he had not the problem on 9.3, it must 
be a more general issue. 
 
Chris, can we figure out somewho which (kernel ?) change did cause the slow 
down ? Anything else what could be tested ? 

*** Bug 114024 has been marked as a duplicate of this bug. ***

How do I interpret the SUSE version 10.1?:
SUSE 10.0 is not even public yet and you have already decided there will be no
fast writing to USB devices for this version? That seems quite a show stopper to me.

it is very strange, 
when i mount in s93 WITH sync speed is about 1.5 Mb/s
on 9.3
trin:/ # cat proc/mounts | grep vfat
/dev/sdb1 /mnt vfat rw,sync,nodiratime,fmask=0022,dmask=0022,codepage=cp437,
iocharset=iso8859-1 0 0
trin:/tmp # ll -h test
-rw-r--r--  1 root root 20M Sep 20 09:25 test

trin:/tmp # time cp test /mnt/
real    0m11.548s
user    0m0.004s
sys     0m0.225s

so speed about 2MB/s 
on 10.0

wintermute:~> cat /proc/mounts | grep vfat
/dev/sdc1 /media/usbdisk vfat rw,sync,nodiratime,nosuid,nodev,uid=10019,gid=100,
fmask=0022,dmask=0022,codepage=cp437,iocharset=iso8859-1,utf8 0 0

wintermute:/tmp> ll -h test
-rwxr-xr-x  1 root root 20M 2005-09-20 09:38 test

wintermute:/tmp # time cp test /media/usbdisk/
real    5m32.887s
user    0m0.024s
sys     0m0.944s
60kB/s it is terible

it is realy sync problem?
and make usb sticks under 10 unusable
could someone check extern hard disk via usb (i do not have any)

i tried (manualy cancled)
wintermute:/tmp # dd if=test of=/media/usbdisk/test bs=1024
10223616 bytes (10 MB) copied, 334,656 seconds, 30,5 kB/s
wintermute:/tmp # dd if=test of=/media/usbdisk/test bs=10240
6656000 bytes (6,7 MB) copied, 84,3576 seconds, 78,9 kB/s

and last but realy interesting result with mount without sync
wintermute:/tmp # cat /proc/mounts | grep vfat
/dev/sdc1 /mnt vfat rw,nodiratime,fmask=0022,dmask=0022,codepage=cp437,
iocharset=iso8859-1 0 0
wintermute:/tmp # time cp test /mnt/

real    0m0.204s
user    0m0.000s
sys     0m0.128s
wintermute:/tmp # time sync

real    0m10.647s
user    0m0.000s
sys     0m0.040s

speed again on ~2MB/s

i borrow usb extern hard disk
on suse 93
trin: mount | grep /dev/sdb1
/dev/sdb1 on /mnt type ext3 (rw)
trin:/tmp # dd if=test of=/mnt/test
40960+0 vstoupivších záznamů
40960+0 vystoupivších záznamů
20971520 bytes (21 MB) copied, 0,497808 seconds, 42,1 MB/s
#since is not neded to run manualy !

with sinc
trin:/tmp # mount | grep /dev/sdb1
/dev/sdb1 on /mnt type ext3 (rw,sync)
trin:/tmp # dd if=test of=/mnt/test   #cancled
665600 bytes (666 kB) copied, 4,64604 seconds, 143 kB/s

on suse 10
wintermute:/tmp # mount | grep usbdisk_1
/dev/sdd6 on /media/usbdisk_1 type subfs (rw,nosuid,nodev,sync,fs=ext3)
wintermute:/tmp # dd if=test of=/media/usbdisk_1/test
1754112 bytes (1.8 MB) copied, 57.3608 seconds, 30.6 kB/s

with switching sync of 
20971520 bytes (21 MB) copied, 0.637884 seconds, 32.9 MB/s

i do not get it 
how is possible that usbstick work great on suse 9.3 while hard drive not?

I think this should not be moved to 10.1. I moved it back to 10.0. For me it's a
really critical bug of 10.0 if users need ages to copy some data to their USB
stick which needed only seconds in SL 9.3. The exchange (writing) rate is about
100 time slower than SL 9.3. It should be a high priority to fix this in 10.0.

I have two USB-memorysticks here. 
One takes 8 seconds for the Megabyte and is  
Disk /dev/sda: 1048 MB, 1048576000 bytes 
64 heads, 32 sectors/track, 1000 cylinders 
Units = cylinders of 2048 * 512 = 1048576 bytes 
 
   Device Boot      Start         End      Blocks   Id  System 
/dev/sda1   *           1        1000     1023983+   6  FAT16 
Partition 1 has different physical/logical endings: 
     phys=(998, 63, 32) logical=(999, 63, 31) 
 
The other takes 2 Minutes per megabyte and is 
Disk /dev/sda: 260 MB, 260046848 bytes 
16 heads, 50 sectors/track, 634 cylinders 
Units = cylinders of 800 * 512 = 409600 bytes 
 
   Device Boot      Start         End      Blocks   Id  System 
/dev/sda1   *           1         634      253527    b  W95 FAT32 
Partition 1 has different physical/logical endings: 
     phys=(990, 15, 50) logical=(633, 14, 4) 
 
 
Does FAT16 vs FAT32 make that difference? 
Both devices write 1MB and umount instantaneously if mounted manually without 
'sync' option. 

I see the same problems with an attached usb stick and with an attached usb 
hard drive. I do not have problems when I mount a encrypted partition manually 
from the same hard drive. 
I had no problems with Fedora Core 4. 
 
Should I provide any data, logs or whatever? 

I have the very same problem with my USB stick on SUSE 10.0, the problem didn't
appeared on SUSE 9.3. Data Transfer rate is about 40 kb/s which is catastrophic.
I don't want to spend more than 1 hour to transfer 200 Mb on my USB key !

If I can help you to resolve this bug, I can send you all infos you want.

I use the workaround shown in comment 19 without any problems for quite a 
while. I believe the workaround is correct. As the drive gets unmounted 
shortly after every write, all data is synced soon enough. 
So, whenever the device stops blinking, I can unplug. This is sufficiently 
safe for me. Yes, (Sync_on_close would still be better.) 

No, it's not a good workaround. I tried it. Ok, when I copy some files it works. But did you try to copy about 100MB of files in your USB stick ? It takes via KDE about 15/20 seconds, but... It become impossible to use the SB stick during sync, one or two minutes... During this time, I'm unable to copy another file on my usb stick, or I'm unable to refresh the konqueror Window, and so.

And, this hack works for only one USB stick, and you have to modify this XML file for every Usb Stick model you wan't to put in your PC ????

Say me if I'm wrong, but in Suse 9.3 sync was enabled, and this was very fast, over 1 or 2 MB/s...

I expect a patch will soon be distributed because this situation is not acceptable. Some web sites become to report it, as Slyuni.org or alionet.org...

(In reply to comment #40)
> And, this hack works for only one USB stick, and you have to modify this XML
> file for every Usb Stick model you wan't to put in your PC ????
 
No, this is not correct. You can change the fdi file to do this for all USB-storage devices. You need only to replace this line (untested):

<match key="volume.uuid" string="==UUID==">   

with

<match key="@info.parent:storage.bus" string="usb">

ok, I will try this, thanks. But the sync problem isn't resolved...

(In reply to comment #41)
> (In reply to comment #40)
> No, this is not correct. You can change the fdi file to do this for all
> USB-storage devices. You need only to replace this line (untested):
>
> <match key="volume.uuid" string="==UUID==">   
> 
> with
> 
> <match key="@info.parent:storage.bus" string="usb">
> 
Thank you very much! This worked well!

maTTeo

*** Bug 121985 has been marked as a duplicate of this bug. ***

Will anybody update referenced SDB article that it is relevant for 10.0 either?

I take a look at the SDB-articel, but should they not merged to opensuse? or or is this a misunderstanding? (I can't see actual more than one articles for 10.0 in SDB)

*** Bug 128627 has been marked as a duplicate of this bug. ***

@Holaf Hering: I did the grep -w sync /proc/mounts, and the result is:

/dev/fd0 /media/floppy subfs rw,sync,nosuid,nodev 0 0
/dev/sda1 /media/usbdisk subfs rw,sync,nosuid,nodev 0 0

I tried to disable the sync and seems to be better. 
Definetly this bug is serious and I don't thing it can be postponed to 10.1!

I know I'll repeat facts, but performance problems sometimes don't show up clear enough on fast devices. Here are the facts for USB 1 (writing to an USB 2 memory stick with "dd"):
~> time dd if=SUSE-10.0-CDs.wav bs=512 of=/media/USBDISKPRO/xxx
201306 bytes (201 kB) copied, 24,3431 seconds, 8,3 kB/s

real    0m24.811s
user    0m0.000s
sys     0m0.104s

8kB over USB 1.0, that cannot be state of the art!!!  Do we want to wait for reviews to point that out, or do we try to fix that?

The opposite direction runs at virtually full speed:
~> time dd of=/dev/null bs=512 if=/media/USBDISKPRO/xxx
201306 bytes (201 kB) copied, 0,205332 seconds, 980 kB/s

Using traditional mount gives at least 268kB/s, over 30 times the speed of the new sync mounts:
# time sh -c "dd if=SUSE-10.0-CDs.wav bs=512 of=/mnt/xxx;sync;sync;sync"
393+1 Datensätze ein
393+1 Datensätze aus
201306 bytes (201 kB) copied, 0,008398 seconds, 24,0 MB/s

real    0m0.750s
user    0m0.024s
sys     0m0.044s

Finally here are the results for a floppy disk:
:~> time dd if=SUSE-10.0-CDs.wav bs=512 of=/media/floppy/xxx
201306 bytes (201 kB) copied, 309,856 seconds, 0,6 kB/s

real    5m11.210s
user    0m0.004s

For comparison, using "mcopy" (which also writes synchronously, inclusing the user's response time to the overwrite question):
~> time mcopy SUSE-10.0-CDs.wav a:xxx
Long file name "xxx" already exists.
a)utorename A)utorename-all r)ename R)ename-all o)verwrite O)verwrite-all
s)kip S)kip-all q)uit (aArRoOsSq): o

real    0m25.421s
user    0m0.004s
sys     0m0.016s

That's about 12 times faster!

(In reply to comment #19)
> I found a documentation which describes how to disable the sync mount options
> of
> a subfs mounted device.
> 
> http://portal.suse.de/sdb/de/2005/06/dkukawka_halnosync_html.html
> https://bugzilla.novell.com/show_bug.cgi?id=85413#c5
> 
> Mounting hotplugged devices without "sync"
> -------------------------------------------
> 
> If writing to USB storage devices like USB sticks or USB hard drive is
> rather slow or if you do not want synchronized writing to these devices,
> disable the sync mount option.
> 
> 1.
> Create the directory /usr/share/hal/fdi/policy/95userpolicy:
> 
> mkdir -p /usr/share/hal/fdi/policy/95userpolicy/
> 
> 2.
> Create a file with the name nosync.fdi and the following contents:
> 
> vi /usr/share/hal/fdi/policy/95userpolicy/nosync.fdi
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <deviceinfo version="0.2">   
>   <device>   
>     <!-- disable sync for mount -->   
>     <match key="block.is_volume" bool="true">   
>       <match key="volume.fsusage" string="filesystem">   
>         <match key="volume.uuid" string="==UUID==">   
>           <merge key="volume.policy.mount_option.sync"
> type="bool">false</merge>   
>         </match>   
>       </match>   
>     </match>   
>   </device>   
> </deviceinfo>
> 
> 3.
> Adjust the line <match key="volume.uuid" string="==UUID=="> according to
> your hardware. Run 'lshal' to retrieve the "volume.uuid" and enter this id
> instead of ==UUID==
> 
> 4.
> Finally restart the HAL service with
> 
> rchal restart
> 

Hi,
I used the settings described in comment 19 and it works! Data rates are now as high as ever. 
One tip: if you looking for the right string for "volume.uuid, you can run lshal with lshal > /root/lshal_out.txt within a console during the USB-Stick is mounted and open the Textfile with kwrite /root/lshal_out.txt, Then you can search within the textfile for the volume,uuid-string according to the mountpoint of your USB-Stick. These are the outputs for my usb-stick.
>>>>>
volume.mount_point = '/media/usbdisk'  (string)
  volume.label = ''  (string)
  volume.uuid = 'E4DE-3250'  (string)
<<<<<

Another tip:
If you want to disable sync-option in every usb-device, you can change the nosync.fdi-file to this:

<?xml version="1.0" encoding="ISO-8859-1"?> <!-- -*- SGML -*- -->   
<deviceinfo version="0.2">   
  <device>   
    <!-- disable sync for mount -->   
    <match key="block.is_volume" bool="true">   
      <match key="volume.fsusage" string="filesystem">   
        <match key="@info.parent:storage.bus" string="usb"> 
          <merge key="volume.policy.mount_option.sync" type="bool">false</merge>   
        </match>   
      </match>   
    </match>   
  </device>   
</deviceinfo>

Copy the lines above to a texteditor and save the nosync.fdi file to /usr/share/hal/fdi/policy/95userpolicy

After "rchal restart" the sync option is disabled for every usb-device.

A comment: I used Suse 9.3 before and didnt had the problems with my usb-stick there. I also find, that this is a very serious bug and should fixed in Suse 10.0.

Greetings 
Steffen

I tried the "Another tip" in Comment 50 to disable the sync-option. No gain in speed was achieved and writing to a USB flash drive is still very slow. I didn't 
change the configuration according to Comment 19 or SUSE 10.0 Release Notes, since I hoped that Comment 50's suggestion will allow not to enter the volume id 
of every USB flash memory device I use. 

And I couldn't find any mention of USB device problems in SUSE Support Database.

Was anybody able to to:
1) disable the sync option for all possible USB memory devices;
2) maintain high (nominal) writing speed?

Thanks.

(In reply to comment #51)
> I tried the "Another tip" in Comment 50 to disable the sync-option. No gain in
> speed was achieved and writing to a USB flash drive is still very slow. 


Sorry, there was a line wrap in the text. I will made my nosync.fdi as an attachment.

Steffen

Created attachment 54777 [details]
nosync.fdi file

Steffen's idea work fine for me. Even for the machine, where my previous 
attempts failed to remove the sync flag.

I followed the guide in Comments 50, 52, 53, and didn't get the high speed anyhow. When I hand-mount the USB flash drive with "mount -t vfat ...", I get very high writing speed. Following the comments of Steffen, I get writing speed 40 KBytes/s.

I think that there are probably different sub-classes of USB devices, and the nosync.fdi in Comment 53 is not effective for my device.

 

THis hack worked correcty for me. Michael, did you :
1 - Put this file in the good directory ?
2 - unplugged/unmounted the usb stick/key/drive BEFORE restart hal ?
3 - restarted HAL (rchal restart) as root after ?

So, if you think you have another subclass of USB device, can you put here the dmesg logs when you put your device plugged and the hald log ?

*** Bug 121985 has been marked as a duplicate of this bug. ***

Yes, I did all in Comments 52, 53, 56. I rebooted the PC without the USB flash drive. I restarted the hal daemon. Didn't work.

Where is the hald's log file? I couldn't find hal.log, hald.log or similar. When I find it, I will attach that log file, hwinfo and dmesg.

Thanks,
Michael.


(In reply to comment #56)
> THis hack worked correcty for me. Michael, did you :
> 1 - Put this file in the good directory ?
> 2 - unplugged/unmounted the usb stick/key/drive BEFORE restart hal ?
> 3 - restarted HAL (rchal restart) as root after ?
> 
> So, if you think you have another subclass of USB device, can you put here the
> dmesg logs when you put your device plugged and the hald log ?

I would prefer to stop this discussion about changing fdi-files. This should not be discussed in the bugzilla, this is not a support forum. Please discuss this on openssuse or suse-linux mailinglist so we can concentrate to a solution of the problem/bug.

Thanks.

Same problem here - write speed of 170kB/s under 10.0, whereas it used to be 5MB/s under 9.3.

Let me recap: It was working with full speed and sync under 9.3, so this should also be possible under 10.0 without any hacks (e.g. turning off sync). At the moment, this bug is really more than critical, as most users will use USB sticks once in a while and with this speed it is just useless.

Let me point You to this post on HAL mailinglists:
http://lists.freedesktop.org/archives/hal/2005-October/003596.html

It is, supposedly, not only about the speed of writing, but also about possible hardware damage....

I'll keep flash-drives away from suse for now     :(

The given solution from comment#19 doesn't work at my private host at home.
The modification, given in comment#41, doesn't work neither.

If I call "mount", I see this:
/dev/sde1 on /media/KLAUS256 type subfs (rw,nosuid,nodev,sync,fs=floppyfss,procuid,utf8=true)

I will attache the output of lshal.

I don't know, if this is related, but I connect my usbstick via a multi-in-many front panel to my Linux box. This is a kind of "usbhub" and only capable of USB-1.1, whereas my motherboard is USB-2.0?

Oops... After downloading nosync.fdi attachment file and diffing it with mine, I noticed that mine missed a line. Inserting this missing line fixed problem at my host. Shame on me. :-(
Sorry for confusion.

On the missing line (but slightly off-topic): I had realized that there is a DTD (Document Type efinition) for such files, but the files themselves don't specify it. Thus XML validation fails due to lack of that DTD. This all means: If tthose files would specify the DTD, and if the DTD is well-written, such an error should have been detected by validating the XML file.

Here is some testing that was done by another user. I asked for permission to add it to the bug and he said that would be fine. Below are the results.

----

My first test was with the 250gb hard drive formatted as ReiserFS Copying 
a 494mb iso image from my internal HD.

/dev/sda5 on /media/testvol type subfs (rw,nosuid,nodev,sync,fs=reiserfs)
linux:/home/jjennings/temp # time cp smartstart-7.40-0.iso /media/testvol/

real    8m24.695s
user    0m0.132s
sys     1m17.989s

So it took 8 minutes to copy the iso image

Next test is with Sync disabled which took 30 seconds

real    0m30.157s
user    0m0.100s
sys     0m8.233s

Final test, I remounted the drive bypassing subfs(USB sub system), the 
copy only took 2 seconds

/dev/sda5 on /media/testvol type reiserfs (rw)
linux:/home/jjennings/temp # time cp smartstart-7.40-0.iso /media/testvol/

real    0m2.638s
user    0m0.044s
sys     0m2.568s


Next I wondered if another Format would work better, plus I am beginning 
to hate ReiserFS with a passion.
Same tests but with Ext3 Format

Sync Enabled
real    36m44.669s
user    0m0.008s
sys     0m0.656s

Sync Disabled
real    0m5.890s
user    0m0.076s
sys     0m5.496s

manual Mount  (1.9 Seconds to copy 495MBs)
real    0m1.973s
user    0m0.072s
sys     0m1.856s

Hello
I'm using KDE 3.4.2 level b on OpenSuse 10
I've this slow transfer rate problem only from my pc to my usbdrive/microdrive (copying or moving file at less than 2Mb/s). In the other direction (usbdrive to my pc) the speed rate is normal (moving at 6Mb/s or copying at 15-16 Mb/s).
On my external usb hard drive the transfer rate are around 80kb/s
My usbdrive is fat32 formated and my pc is reiserFS formated.
Is it possible that there's a link with this other bug : 117945 ?
Also, kdf don't estimate the % of occupation of my usbdrive (seems not mounted).
I'd no problems at all with ubuntu/kubuntu breezy badger and mandriva 2005.

I hope these details could help you to resolve the usb probs.

Thank you.

*** Bug 132831 has been marked as a duplicate of this bug. ***

*** Bug 132631 has been marked as a duplicate of this bug. ***

This bug also affects my usb 1.1 flash drive. I am getting 30 KB/s and sometimes it even stalls. I have tried both ports on my computer under Linux it doesn't work. Funny thing though the first time I installed OpenSuSE 10 the usb drive worked perfectly, then one day POOF gone. I tried reinstalling OpenSuSE but that didn't help. I tried my drive under windows, it works perfectly.

This bug also affects all my usb storage devices (usb flash disk (usb1.1 and 2.0) and external harddisk (usb2) (opensuse10.0 final on intel centrino chipset)

Interestingly, if the file system is changed to for example ext3 or
reiser. the transfer rates are normal !! (both on external harddisk as usb flash device). 
I am curenly using an usb flash device with an ext3 file system and this works fine.

The only thing i noticed on my external hardrive was a temporal lock-up of konqueror when using the device. It seems that the device is unmounted and mounted again quickly (This behaviour is similar as found with Suse 9.2).
Notably, the vfat filesystem is not accessible for the user accounts. This was not the case in suse 9.1, 9.2 and 9.3 !! 

In my humble view the problem seems to link to the fat/vfat file system. At least for the the external devices i tested (both usb1.1 and 2.0 on an Usb2 port).

I hope this info helps to fix the problem.

If that helps any, I have a (VFAT formatted) USB stick here that shows the very same problem with 10.0. I get data transfer rates in the order of 150 kB/sec with 10.0 and sync. On 9.3 and 9.2 without sync it used to be about 13 MB/sec.

My external USB HD on the same machine (formatted with ext2) drops from normally 20 MB/sec to about 13 MB/sec - i.e. to about 60% of the original performance, not like 0.1% like the USB stick.

This behaviour makes that USB stick unusable with that system.

It seems that up to now there is no comfortable way to operate the USB flash drives under 10.0.

I have found that the only adjustment that helps to regain the writing speed is the one described in the Release Notes (file = /usr/share/doc/release-notes/RELEASE-NOTES.en.html

In addition, I have found that the following works the best (or doesn't work):

1) Before performing the instructions in Release Notes, create in advance and change the ownership (not only the read/write/execute mode) of directory /media/usbdisk to the user who will use the device. If there are several filesystem types, do the same with, e.g., /media/usbdisk_fat

2) After performing the instructions in Relaese Notes, reboot the computer: restarting the daemon is not enough.

3) Use reiserfs for highest speed (a modern USB flash drive gives up to 40 MBytes/s maximal writing speed and about 5 MBytes/s sustained speed).

4) For fun, I have tried installing SUSE on the USB flash drive: it began, but hung after 300 MB was written to the USB drive.

5) Partitioning into more than two or three partitions doesn't work properly.

Regarding comment #86: Please: The issue is about "sync mounts are very slow", and not: "sync mounts are used by mistake". This all doesn't fix the problem of slow sync mounts.

Like many others I too have suffered through this bug.  It is a very critical
bug.  I have several Suse machines and implementing the workaround on all of
them would be very tedious.  Also, I recommend Suse to a lot of people.  This
bug rather shakes my confidence in Suse.  It has been 4 months since it was
submitted and a fix has not yet been released for Suse 10.0?  

Ulrich, nobody has shown that sync mounts can be as fast as async mounts. I fear, they can not. We seek to get back to *full* speed, while retaining the needed safety. I would not care, if the end result is called a sync mount, or a just-do-it-right mount.

On comment #89: This report also is not about the fact that sync mounts are slower than async mounts. THIS REPORT IS ABOUT THE FACT THAT SYNC MOUNTS ARE AT LEAST FIVE TIMES AS SLOW AS THEY WERE IN SUSE LINUX 9.2 or 9.3.  Please put a floppy into your drive and save a 200kB file to it. Do't use an empty floppy, but use a heavily used one. The floppy head will move around like crazy while you wait and wait. Compared to USB the advantage is that you can hear what's going on.

Ulrich, please note:
* sync on 9.x was nearly so fast as async, because this was simply not implemted as sync. sync worked not in this kernel version. Now its correct implemented in the kernel from 10.0 and it's slower than "before".
* we work on this issue, but's not so simple as some people maybe think

OK, what are the semantics of the "sync" option now? Is it O_DSYNC (data written syncronously, but not necessarily file attributes), O_SYNC (R_DSYNC with attributes being written synchromously as well? I'd wish for a O_CSYNC (sync all file data and attributes when a file is closed).

-osync means that any time on-disk data is changed, it is written immediately. This applies to data and metadata alike.

We've been experimenting with a few different solutions for this problem, including the -osync_on_close (your O_CSYNC). The problem is that for a work load of say, "cp /usr/lib /media/usbdisk" where usbdisk is a flash device, the performance is still quite a bit worse than with -oasync. Further, in order to be truly safe, it needs to be also mounted with -odirsync (implied with -osync, so you wouldn't see it now), so that things like deletes and renames are still handled safely.

This is an issue we take very seriously, and we're trying to find a workable solution for all the most common workloads.

Obviously I have this problem too, however only on a vfat connected memory stick and a vfat connected MP3 player.  My DVD writer (USB Connected) and my 120GB HDD (USB Connected with ReiserFS) do not experience this problem.  However I have what I believe to be a related problem, so I'm putting it in here as I think it should be considered WITH the fix for this problem.

What I have noticed from Suse 9.2, 9.3, and now 10, is that externally connected ReiserFS and NTFS devices (therefore filesystem independant) using submount continuously sync something with the device, slowing or 'pausing' the whole system about every 10 seconds or so.  I can connect the external HDD devices via USB or Firewire with the same results.  I expect it is something to do with the sync option.  I would point out this has happened on various hardware and is not in my opinion related to the hardware in any way.  

It's interesting that in Suse 10 this seems to only happen if I open a konqueror window and browse to the drive and leave it open (although I'm sure there are other things I could open this probably just means it's not actually mounted until you connect something to it), in previous versions of Suse it happened after simply connecting the device and then doing nothing.  

The fix for me is to use console and cd into a directory and type ls once and leave the console open, it appears this tricks the drive into thinking it's in use and therefore doesn't attempt to run this sync option (if that is what is happening) and therefore runs like you would expect, as I said I only have performance issues with the VFAT submount connected flash devices.  Annoyingly I have to do this for every submount connected device, so when you have a USB/Firewire DVD Writer, two USB/Firewire connected Hard Disks, and two vfat connected USB flash memory devices it's obviously an amazingly large annoyance.

I figured if you are fixing a sync issue of some sort it would be important to resolve this with it and that it may also be a part of the problem.  Obviously I'll leave that call up to someone who knows more about the technical aspects of submount.

Thanks!

Please fix this before 10.1!
MP3 players and digital cameras aren´t really usable with SuSE ATM

*** Bug 141106 has been marked as a duplicate of this bug. ***

I have done my 95userpolicy.fdi as:

/usr/share/hal/fdi/policy/20thirdparty/95userpolicy.fdi, rather than in the directory above that, and it seems to work well. Is there a reason folks didn't place it under 20thirdparty?

Increasing the logical sector size in generating the fatfs will increase speed:

LSZ=512 results in ~ 20kb/s
LSZ=4096 results in ~120kb/s 

usb1 device on an USB port

Moreover, the speed differences are connect to specific hardware, dpending on the attached device i recorder between 20 and 360 kb/s all usb1 devices and all with fat16.

i hope this info helps

 

This bug is still in SUSE 10.1 Beta2. When I wrote on my USB stick the speed was very bad. Only a few KB/s...

If you want to have some information about my USB stick I can attach more infos.

Felix: KDE? GNOME? /proc/mounts?

Stephan: Beta3

Right now under Gnome. Copied a 8 MB ZIP file in Nautilus. Stopped after 40 s, it still showed me 25 minutes...

Same behaviour when copying in Gnome terminal, stopped after more than one minute.

As a reference: mounted manually without sync: about 10 s. That's also not very fast but much, much faster than with the sync option.

See attachements.

lsusb -v for my USB stick and the proc-mounts

Created attachment 66449 [details]
cat /proc/mounts

Created attachment 66450 [details]
lsusbt -v for my Alcor Micro USB stick

My Trekstore USB hard disk is very fast but it's NOT mounted with sync option in Gnome.

Copy 8 MB ZIP without sync: 1-2 sec
with sync (manually mounted): 7 sec

Don't understand why my memory stick IS mounted with sync option but /dev/sda1 IS NOT mounted with sync option although it is formatted with FAT.

Maybe it's because I have also two EXT3 partitions on it? see attachement proc-mount-trekstore.

Shall I open a bug report that a Stick with one partition is mounted WITH sync while a USB disk with multiple partitions is mounted WITHOUT sync?

Created attachment 66451 [details]
cat /proc/mounts for my Trekstore USB hard disk

SL10.1 has a new mount option for fat, which should get used by default by the gnome and kde automounting tools.  This should use -o flush instead of -o sync, which will be much faster.  -o flush is only available for fat, but it will start all io immediately without waiting for it to finish.  This means you have to wait for io activity to stop before pulling out the disk/stick (look for the flashing light).

So, the disk in comment #103 should have been mounted with -o flush.  Coolo?

With my installation I get different mounts with KDE and Gnome. I attached the /proc/mounts in KDE, too.

In Gnome I had the sync option and in KDE I had no sync and no flush option - see attachements.

But maybe it's my fault - had problems with Beta3 installation: it stopped after CD1 because of a LVM problem. So I installed everything by hand from CD2. I have seen no KDE nor Gnome directories in /etc. As far as I know are there the default configs for both DE's. So maybe that's why I got these different results!?

I will wait till next Beta and post bugs then with a clean installed system ;)

Created attachment 66453 [details]
cat /proc/mounts in KDE

The missing flush option in KDE is #145851 - will be fixed with beta4 I assume. The GNOME default are still sync, but depend on #145851 too. But for KDE this bug is fixed (as no sync mount anymore :)

(In reply to comment #108)
> SL10.1 has a new mount option for fat, which should get used by default by the
> gnome and kde automounting tools.  This should use -o flush instead of -o sync,
[...]
(In reply to comment #109)
> With my installation I get different mounts with KDE and Gnome.

I think it's poor design, to make the GUI components decide on how the media is going to be mounted.

Before this problem is definitely fixed with a patch for SUSE 10.0 (hopefully) or in SUSE 10.1, I suggest to change again the release-notes and add the possibility to mount ALL usb storage devices without the sync option.

I mean, add what has been suggested above (comments #41 and #50, attachment "nosync.fdi file") to the release-notes in order to enable "normal" users (who do not search through bug reports) to use their usb devices with high speed.

I have had the problem of writing speed to usb storage devices on different computers (PC and notebook) and on different devices (a memory stick and a compact-flash card in a card reader). By adding a file
/usr/share/hal/fdi/policy/20thirdparty/95-storage-nosync.fdi
with the suggested content

<?xml version="1.0" encoding="ISO-8859-1"?>
<!-- disable sync for mount -->
<deviceinfo version="0.2">
  <device>
  <match key="block.is_volume" bool="true">
  <match key="volume.fsusage" string="filesystem">
  <match key="@info.parent:storage.bus" string="usb">
    <merge key="volume.policy.mount_option.sync" type="bool">false</merge>
  </match>
  </match>
  </match>
  </device>
</deviceinfo>

and calling "rchal restart" (I did not have to restart the computer!), now my usb devices are mounted without the sync option and the writing speed is as fast as experienced under SUSE 9.2 before.

I have also written several MB to the usb memory stick and plugged it off immediately after its LED stopped blinking. After plugging it in again I compared the file on the stick with its copy on the hard disk, and they were identical, so everything was written without problem.

I understand that you want to be sure about data integrity before installing a patch for every SUSE user. But I really suggest to offer at least this possibility to everyone by adding the above suggestion for all usb devices to the release-notes.

For 10.1, we'll have the flush option.  At this point the functionality is more or less done.  This bug has become rather long and difficult to follow, I would like people to open new bugs for problems they see in the 10.1 betas.  Thank you all for your input, and for helping us work through things.

I know the current solutions are not perfect, but I'll be adding support for more filesystems to use -o flush in later releases, and we'll be able to fine tune our subfs-free implementations over time.

In freshly installed SUSE Linux 10.1, the USB flash drive is again mounted with sync option.

I have tested this on two PCs. FAT partition of the memory stick is mounted with flush option and has very high write speed. But the reiserfs partition is mounted with sync option, and has write speed 400 kbytes/s.

This is the output of "mount":

/dev/sdc2 on /media/usbdisk type reiserfs (rw,nosuid,nodev,sync)

/dev/sdc1 on /media/usbdisk-1 type vfat (rw,nosuid,nodev,noatime,flush,uid=1000,utf8,shortname=lower)

Unmounting and manually mounting the reiserfs partition, as in SUSE Linux 10.0, allows writing with 20 Mbytes/s.

As #114 states: "I'll be adding support more filesystems to use -o flush in later releases". Later as in after 10.1

So if you use KDE, go into sysinfo:/ and change the mount options for your stick

Silly me, but one would think that KDE's sysinfo:/ for usb sticks would offer all the options available for the filesystem type on the stick. My stick is reiserfs, and yet I see no way to do acl and user_xattr (even when messing with .kde/share/config/mediamanagerrc manually).

Or am I just expecting too much?

I've also tried adding options for acl and user_xattr to the 95-storage-nosync.fdi file under /usr/share/hal/fdi/policy/20thirdparty/ like this:

<?xml version="1.0" encoding="UTF-8"?>
<deviceinfo version="0.2">
  <device>
    <!-- disable sync for mount -->
    <match key="block.is_volume" bool="true">
      <match key="volume.fsusage" string="filesystem">
        <match key="@info.parent:storage.bus" string="usb">
          <merge key="volume.policy.mount_option.sync" type="bool">false</merge>
          <merge key="volume.policy.mount_option.acl" type="bool">true</merge>
          <merge key="volume.policy.mount_option.user_xattr" type="bool">true</merge>
        </match>
      </match>
    </match>
  </device>
</deviceinfo>

but it doesn't work :-(. Probably since KDE isn't using it?

--Jon Robison

Hi Jonathon -

This is a separate issue than the one for which the bug was originally filed. Can you open a new bug describing this problem, please?