Samba - Move directory between two discs

[robw]

Hello,
I've the following problem:
I want to move a directory from one Samba-disc to another - does not work in any case.
I have all rights on the shares; I can copy or delete a directory, I can create a new directory and so on...

I use Salamander 3.02 (Windows 7 Pro) to move files and directories between two NTFS-discs mounted with NTFS-3g driver on a SAMBA-share on Debian-Linux. Umask = 0000, means every permission for everyone (see attached image)) . When I want to move a whole directory from one to another share (means from one to another disc) it fails. Creating the directory on the target disc and then copying the files is OK. Moving the whole directory with windows-explorer is also OK! It's equal, if I use Salamander(x86) or Salamander(64bit).

It's a bit annoying to use Explorer or to make the tour: Create directory -> Copy the files -> delete the source files. I'm sure, it's not a problem of user privileges, since in Explorer it works. Any suggestion?

Many TiA
robw

[tukanos]

Does Samba log says something? Maybe you need to change the log level there. It would be helpful if we could see the log.

[robw]

Thank you tukanos for your reply, but there ist nothing useful in the log even with level 3 or 4.
The curious point is, if I want to *move* the directory "Test" from one disc to another disc it fails.
If I *copy* the directory, everything goes OK.
If I first create a directory "Test" on the target disc (goes OK!) an then *move* the directory "Test" from source disc there, it goes OK too!
In latter case I get a warning message in samba log:

Code: Select all

[2014/08/31 10:53:48.043160,  2] ../source3/smbd/open.c:3053(open_directory)
  open_directory: unable to create video/Serie/Test. Error was NT_STATUS_OBJECT_NAME_COLLISION

but this is not offending a good finish. Salamander (Samba) joins the two directories.
In failing cases of moving the directories I get no warning messages at all.

robw

[tukanos]

That looks like a Samba configuration issue. The obvious issue is that move does not create the directory as it should - that should produce samba log output as it is a deny of an operation.

You have to remember that linux is case sensitive and I don't know if you Samba setup is case sensitive or not. What can be also an issue can be if you have different encodings of the filesystems. According to SMB - MS your issue (the log has written) is - 0xC0000035
STATUS_OBJECT_NAME_COLLISION - The object name already exists. (from http://msdn.microsoft.com/en-us/library/cc246324.aspx)

The object name already exists.

What kind of filesystems do you have on the samba boxes?

[robw]

Hello tukanos, thanks for your advice!
The case of the directory-name is no issue, since my Samba is configured case-insensitive. The two samba boxes are external USB-NTFS-discs formatted with native NTFS and mounted with NTFS-3G (fuse-driver) with umask 777 which means: everyone is permitted every operation.
The object_name_collision is no issue too i think, because joining the already existent directories operates flawless!

In windows-explorer the operation is no problem at all - look at my screen pics (If i could upload a small flv you could see it in motion). You'll recognize that the flawless move operations of Explorer (above) are displayed in Salamander panes at same time too. Only Salamander does not create the directory as it should as you mentioned already. This is my opinion too. Hence I don't think that it's a samba issue.

Greetings robw

[tukanos]

Hello robw,

I still think the core of the problem could be the NTFS-3g driver. I have multiple Samba boxes even over VPN with native filesystems and it works like a charm.

Could you do the following tests?

- map the network path to a disk? (net map...) then try to move the files

- try to switch off async alg. in Salamander -

async_transfer.gif (25.58 KiB) Viewed 17422 times

- try it with native filesystem in Linux without fuse driver in between ntfs-3g and kernel -> ext4 would be optimal. There can be issues with the umask validity on non-native filesystem.

- the move systemA -move-> PC -move-> systemB does work?

- Yes I know that explorer works the question is what is actually done there. I don't know if there is a way to force explorer to give you exact log of what it is doing.

It can actually be both I think setup with samba and NTFS-3g driver is usually quite troublesome.

Cheers,
tukanos

[robw]

Hello tukanos,
here is the result of my tests:
1) mapping the different ntfs-3g discs to different drives in Salamander results in moving the directories works!
2) switching off the async copy method has no effect - does not work.
3) moving the directories between ext-4 disc and ntfs-3g disc over samba-share doesn't work too, equal in which direction. Except they are mapped under different drive letters (Case 1)
4) without fuse-driver (ntfs-3g) I can't write to ntfs disc.

It seems this is an issue dealing with the factor that the two discs are mapped together with linux-symlink under one samba-share.
(BTW: Samba is configured with follow symlinks = yes, wide links = yes, unix extensions = no, force directory mode = 0777 etc.)

5) One further test shows: it must be a bug in AS, the unloved (for me) TotalCommander has no problems with moving a directory from one NTFS-3g box to another... I wouldn't have guessed.
greetz robw

[tukanos]

It seems this is an issue dealing with the factor that the two discs are mapped together with linux-symlink under one samba-share.
(BTW: Samba is configured with follow symlinks = yes, wide links = yes, unix extensions = no, force directory mode = 0777 etc.)

...move files and directories between two NTFS-discs mounted with NTFS-3g driver on a SAMBA-share on Debian-Linux.

I still did not get clear/complete picture of your configuration. You have one Samba share with two external drives (both NTFS-3g).

e.g. (Both are mounted automatically (boot or auto-mapper) or manually?)
/dev/sda1 mounted as /mnt/diskA (-t ntfs-3g)
/dev/sdb2 mounted as /mnt/diskB (-t ntfs-3g)

then you have a samba shared directory /mnt/samba_share. (Could you paste your smb.conf?)

Under the samba_share you have two symlinks (ln -S...) e.g. sym_diskA and sym_diskB? Why don't you have two samba shares pointing to the mounts directly?

addum 1) good that would be a workaround ...
addum 3) I did not mean ext4 <-> Ntfs-3g but trying on some other drive ext4 <-> ext4 transfer with the same configuration
addum 4) That is true. What I meant, is that you should always try to setup samba shares with ext4/ext3/etc. partitions as it always is better solution and gives less spaces for problems.

5) One further test shows: it must be a bug in AS, the unloved (for me) TotalCommander has no problems with moving a directory from one NTFS-3g box to another... I wouldn't have guessed.

It may not be necessarily a bug in AS, it can be lacking a feature - following symlinks or interpreting symlinks in your case.
I think it has different view of the symlinks you have created. The point is that if the drive is mapped then symlinks are viewed differently than with direct access.

tukanos

[robw]

You have one Samba share with two external drives (both NTFS-3g).
e.g. (Both are mounted automatically (boot or auto-mapper) or manually?)
/dev/sda1 mounted as /mnt/diskA (-t ntfs-3g)
/dev/sdb2 mounted as /mnt/diskB (-t ntfs-3g)
then you have a samba shared directory /mnt/samba_share. (Could you paste your smb.conf?)
Under the samba_share you have two symlinks (ln -S...) e.g. sym_diskA and sym_diskB?

That's exactly what I've done. Mounted in fstab to a specific directory in "/home" and symlinked to "/home/daten/media" resp. "/home/daten/backup".
Why? I found it more comfortable to have only one share. The more so as symlinked dirs coming from one disc doesn't make any prob (equal whether its a ext-3 or ntfs) and you can copy and paste from one to another. Only the moving of a dir in AS is quirky.

Here's my config:

Code: Select all

fstab
------------------------------------------------
UUID=260A14470A141681	/home/media 	ntfs	defaults,noatime,big_writes,uid=33,gid=1001,permissions	0	0  # results in /dev/sdb1
UUID=F28CA4858CA445C7	/home/backup	ntfs	defaults,noatime,big_writes,uid=33,gid=1001,permissions	0	0  # results in /dev/sdc1

Code: Select all

smb.conf
---------------------------------------------
[daten]
 path = /home/daten
 comment = Daten
 writable = yes
 browsable = yes
 create mode = 0777
 directory mode = 0777
 force create mode = 0777
 force directory mode = 0777
 force user = www-data
 force group = samba
 force security mode = 0777
 force directory security mode = 0777
 follow symlinks = yes
 wide links = yes
 unix extensions = no
 inherit permissions = yes

As I mentioned before - these drives are external USB-Discs, which should also be readable by a windows-box. Thus I didn't want them as ext-3. Furthermore I've the prob, that I wanted to split my directory for the linux video-disc-recorder (VDR) into two discs due to capacity-problems. To have this recording directory split into to mappings is a littlebit annoying.
Btw. I'v tested my config with FreeCommander too. It moves my dirs too without complain. If this is not a bug in AS this feature is at least a non-expected-behaviour...
Greetz robw

[tukanos]

Btw. I'v tested my config with FreeCommander too. It moves my dirs too without complain. If this is not a bug in AS this feature is at least a non-expected-behaviour...

You are, of course, right that this deficiency should be fixed. The issue here for us to report it is to give the AS developers a log where this does fail - I don't know how to generate it from AS in this situation. There are more linux features I would like to see in AS - like hiding unix/linux based files starting with '.' and some others. (Why I tend to differentiate between bug & missing feature? If the code is missing you can't have bug within.)

Back to your problem. I see your use case scenario now.

Maybe a nice solution would be if you would take /home/daten and change your mount points in fstab /home/media && /home/backup to /home/daten/media && /home/daten/backup. You would circumvent the symlinks and have the same structure.

As I mentioned before - these drives are external USB-Discs, which should also be readable by a windows-box

You can read ext/2/3/4 in windows using Paragon's free utility - https://www.paragon-software.de/de/home/extfs-windows/

Just small note - watch out when using NTFS under linux as linux can't repair the filesystem when some error happens. Since you are recording some TV programs and you have huge data flow I recommend using native filesystems - you would get better speeds and possibility to fix it there. With NTFS you should always check it with 'chkdsk' before using it on windows.

Ein shoenes Tag,

tukanos

[robw]

Hello tukanos,
you're right, ext-3/4 would be a more sincere solution, but now - with about 6TB data - I've no time and nerve to move all that stuff to new discs. The more so as speed is not my bottleneck since it's USB-3. And since the biggest disc I can afford is 4 TB I will have the problem to split the recordings to two discs in any case.

I will test the approach with new mountpoints and hope that the developers of AS will adopt this problem. It's a behaviour of AS since version 1.52...

Many thanks for your patience - greetings robw

[tukanos]

Hello tukanos,
you're right, ext-3/4 would be a more sincere solution, but now - with about 6TB data - I've no time and nerve to move all that stuff to new discs. The more so as speed is not my bottleneck since it's USB-3. And since the biggest disc I can afford is 4 TB I will have the problem to split the recordings to two discs in any case.

I will test the approach with new mountpoints and hope that the developers of AS will adopt this problem. It's a behaviour of AS since version 1.52...

Many thanks for your patience - greetings robw

Glad I could help . Do try the recommended mount points as it should solve your problem. Well AS should fix it hopefully soon but they have higher priorities right now.

If USB-3 is a bottleneck, try this trick. For writing to a drive have a eSATA and for reading have USB-3. This should give you about 20% performance boost.

Take care,

tukanos

[robw]

Once again -
In the meantime I use Salamander 3.06 (Windows 10 Pro)
I've ever the following problem:
I want to move a directory from one Linux-disc (Samba or not) to another - does not work in any case.
I have all rights on the shares; I can copy or delete a directory, I can create a new directory and so on...

When I want to move a whole directory from one to another share (means from one to another disc) it fails. Creating the directory on the target disc and then copying the files is OK. Moving the whole directory with windows-explorer is also OK! It's equal, if I use Salamander(x86) or Salamander(64bit).

It's not only a bit - but very, very! - annoying to make the tour: Create directory -> Copy the files -> delete the source files. I'm sure, it's not a problem of user privileges, since in Explorer it works. The only workaraound is Total Commander! But I don't love it. TC a lot better than AS.

One Year no solution. This cannot be true!

robw

[Jan Rysavy]

We are using Debian and Samba every day with Salamander and moving files works fine here (just tested to be sure).

What Linux distribution and version are you using?

What error message does Salamander show, please post screenshot.

Could you move just one file? Could you move just one directory?

[robw]

Hello Jan, thanks for your reply!
As I've mentioned before it's the same behaviour today. Running Salamander 3.06 in Windows 10 as Administrator.
I have two USB-Discs with NTFS mounted in Debian Jessie. One with my picture-files, the other one as backup.
Access from Windows via Samba 4.1.17.
Copying between the two discs works very fine, equal with files and also folders.
But moving only works for files, not for folders.

movedir1.jpg (28.28 KiB) Viewed 15176 times

movedir2.jpg (33.56 KiB) Viewed 15176 times

Since the discs are formatted with NTFS this cannot be an issue with linux privileges - all are granted: 777!
It complains about "No Access granted" (German: "Zugriff verweigert").
In Total Commander it works. Look:

Total Commander moving

movedir3.jpg (51.75 KiB) Viewed 15178 times

It's funny, because if the directories to move are on the same disc, it works also in salamander. But moving dirs which are on two different discs do only in Total Commander, not in Salamander.
(Btw.: I'm working with your software since early times of 1.52 and love this excellent piece of software, thank you very much!)
But now, I have no clue about this behaviour.
TIA - Greetings robw