‘Well it’s only passing mv a list of–’ yeah yeah yeah, I know, and that’s why I’m calling bullshit. It should be massively harder to execute filenames. Even if 1970s decisions make that the eternal hideous default: the lack of any idiot-proof standard workaround is incomprehensible.

StackOverflow’s full of competing one-liners and people pointing out how each one is considered harmful. The least-skeezy options use exec. That sentence should make anyone recoil in horror.

This is not a filename problem. This is a tool problem. If a single printable character is going to silently expand into a list of names, then for god’s sake, having it put each name in quotes should be fucking trivial.

  • @[email protected]
    link
    fedilink
    4
    edit-2
    5 months ago

    This is not standard behavior. Got links to StackOverflow?

    This works just fine in bash and zsh:

    encoded_funky_name=IUAjJCVeJiooKVtde30nIjo7IGxzOyAmYW1wOyA8Plx8YH5gJChscykke1BBVEh9ISF8fCYmLmpwZw==
    # decodes to: !@#$%^&*()[]{}'":; ls; & <>\|`~`$(ls)${PATH}!!||&&.jpg
    
    mkdir temp
    cd temp
    mkdir temp2
    
    touch "$(echo "${encoded_funky_name}" | base64 -d)"
    
    mv ./*.jpg temp2
    ls -lR
    
    mv temp2/*.jpg .
    ls -lR
    

    Note that it is recommended to use “./” before a raw * in globs to avoid filenames beginning with “-” being interpreted by the command as special arguments (this is not necessary in the above example, but it is a good habit). See https://dwheeler.com/essays/filenames-in-shell.html#globbing for more info. e.g.:

    touch ./-laR 
    
    # Bad ls command:
    ls *
    
    # Good ls command:
    ls ./*
    

    having it put each name in quotes should be fucking trivial

    This is effectively what happens, except that it’s actually more robust than that, since it also accounts for names with quotes and other special characters. Glob expansion does NOT output file names into the shell; it passes them as arguments to the command with no further shell processing. Arguments can contain any special characters.

    Not sure why you’re seeing this behavior. Are you doing something with these file names before passing them to mv? Storing them in variables? Saving them to files?

    • @mindbleachOP
      link
      25 months ago

      It’s literally just mv *.jpg /mnt/Example/Pictures.

      Multiple StackOverflow answers take this as read.

      mv ./*.jpg /mnt/Example/Pictures works fine, though, so thank you for the practical solution. I’m still disgusted there’s any filename that gets treated as… not a filename. That broken stair might be older than I am.

      • @[email protected]
        link
        fedilink
        4
        edit-2
        5 months ago

        Ahhh. That’s a confusing thread, with a couple confounding factors.

        The OP in that thread is attempting to a batch rename, which is out of scope of the mv command.

        If you are NOT trying to change the name of the file, I recommend always making the last argument of mv your destination directory, rather than the destination file. This is strictly mandatory if you are moving multiple files.

        OP in that SO thread used the filename as the destination path — a filename which did not exist, and therefore could not be resolved by the wildcard. And if it had existed, the move would have failed. Because of that, mv /data/*/Sample_*/logs/*_Data_time.err /data/*/Sample_*/logs/*_Data_time_orig.err is not a valid command.

        OP cannot accomplish what they want with a single mv command. They would need to use a loop or something like find.

        I agree that the lack of universal consistency in how commands handle arguments is unfortunate. There’s no way around that without reinventing the entire OS. Many commands let you pass -- as an argument to signify that any subsequent arguments should not be interpreted as options even if they begin with -. But that’s entirely up to the implementation of each program; it’s not a shell feature.

        Future OS designers: consider typed arguments. Abandon all legacy cruft.

        • @mindbleachOP
          link
          15 months ago

          There’s no way around that without reinventing the entire OS.

          Disagree. If wildcard expansion is not a feature of the tool, it shouldn’t be up to the tool to shape how each expanded input is passed. Some hideous escape syntax ought to force *.jpg to appear as "-a.jpg" instead of -a.jpg, once it reaches mv.

          Instead * is acting as a cheeky little ls alternative, minus any degree of formatting control, and piping actual ls input into mv is similarly fraught with complications.

          • @[email protected]
            link
            fedilink
            25 months ago

            Some hideous escape syntax ought to force *.jpg to appear as “-a.jpg” instead of -a.jpg, once it reaches mv

            The issue here is that there’s no difference here between -a.jpg and "-a.jpg".

            Go ahead and try these commands. They are 100% equivalent:

            ls -laR
            ls "-laR"
            ls '-laR'
            

            The reason these are equivalent is because quoting (and expansion, and globbing) is processed in the shell. The shell then passes the evaluated string as an argument to the tool. All of those evaluate to the same string, so they look exactly the same to the ls command.

            What is the solution if there’s a file name that matches something the tool thinks is an argument? What would the correct behavior be?

            The only solution is to include some way to communicate to the tool what each argument is. In practice, you could do ls -- * and it would work even if the first filename was “-laR”. However, not all tools use the -- convention. This is really core to the design of the OS, in that command arguments have no type.

            • @mindbleachOP
              link
              -15 months ago

              Jesus. That’s beyond Javascript levels of “helpfully” reinterpreting strings. That’s borderline Excel behavior.

              What is the point of strings allowing every character besides \0 if they’re gonna eat quotation marks?

              • @[email protected]
                link
                fedilink
                3
                edit-2
                5 months ago

                All I can tell you is that, in my opinion, it’s ridiculous and terrible that old-school terminals haven’t been replaced yet with something more user-friendly and self-explanatory, at least in the same-machine user space. But given that they are what they are, some basic understanding of what shells do is required in order to use them, and you don’t have that understanding (I don’t fault you for this).

                The key point here is that programs/commands always receive an array of string arguments, and it’s the shell’s job to translate your command line into that. Quoting (like in -m="my message"), variable expansion (like $HOME) and various other things are processed by the shell and not the program, and the expectation is that the user knows this. So quotes are never visible to programs, and the upside is that programs never need to process such quotes - making these features universal without having various random bugs in each program’s implementation.

                • @[email protected]
                  link
                  fedilink
                  25 months ago

                  This is exactly correct. This is why I say it would require a whole new OS with no regard for compatibility with current systems.

                  As long as arguments are typeless and transmitted in a single array, there is no universal way to make programs distinguish between an option and file name that are identical. Full stop. No way. This has nothing to do with quoting schemes.

                  There are conventions to work around this (like --) but that is at the command level, NOT the shell level. It is not universal.

                • @mindbleachOP
                  link
                  15 months ago

                  I’ve been using command-line programs for twenty-five years. “Basic” is not the issue, here. This is obscene edge-case behavior for what honestly should be a Hello World level example.

                  Thoroughly explicable causes do not make the outcome any less of a problem.

    • @[email protected]
      link
      fedilink
      0
      edit-2
      5 months ago

      Note that it is recommended to use “./” before a raw * in globs to avoid filenames beginning with “-” being interpreted by the command as special arguments

      Jackpot! I think! Based on OP’s reply to me, this is almost definitely the real problem:

      mv *.jpg /mnt/Example/Pictures

      Where it will then die with an error like mv: invalid option -- '1' depending on the contents of the current directory.

  • @[email protected]
    link
    fedilink
    25 months ago

    I have two questions:

    1. Is this some recent news that you expect us to have heard of? Your post demands context that is not easily understood (but it’s possible to understand through multiple reads)
    2. What shell are you using?

    I’m using bash (GNU bash, version 5.1.16(1)-release) and don’t observe this behavior:

    $ ls
    'first file.txt'  'second file.txt'
    $ cat 'first file.txt' 
    this is originally named "first file.txt"
    $ cat 'second file.txt' 
    this is originally named "second file.txt"
    $ mv *.txt
    $ ls
    'second file.txt'
    $ cat 'second file.txt' 
    this is originally named "first file.txt"
    

    It’s possible that whatever shell you’re using has a bug, but it’s definitely not universal. It’s also possible that this bug only affects builtins like mv and not non-builtin commands.

    • @mindbleachOP
      link
      15 months ago

      I expected people have tried to move files… using the move command.

      I did not expect I’d have to specify that there’s a destination, in an explanation of what *.jpg does, not an explanation of what mv does.

      For clarity:

      GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)

      mv *.jpg /mnt/Example/Pictures

      Where it will then die with an error like mv: invalid option -- '1' depending on the contents of the current directory.

      Every search about this has led back to this StackOverflow page sooner or later. All answers address wildcard expansion.

      • @[email protected]
        link
        fedilink
        1
        edit-2
        5 months ago

        Can you run printf %s\\n *.jpg in that same directory and share the result? I’m really intrigued. If there’s private information in there, I’d be satisfied with just a handful of lines showing the issue with private info replaced with ****s

        Edit: also run alias mv to check whether you have an alias messing things up

        • @mindbleachOP
          link
          25 months ago

          bash: alias: mv: not found

          printf %s\\n *.jpg is just a list of filenames on newlines. There’s 75,000 of them. You’ll have to take my word for it.

          Fortunately, GenderNeutralBro’s aside about using ./* instead of * works as expected. Could have been files beginning with _-. None begin with -, exactly.

      • @FigMcLargeHuge
        link
        15 months ago

        When I run into situations like this, I use the commands that work to write out a script. Eg, in your case the wildcard isn’t working with the mv command, so do something like this:

        ls -1 *.jpg | awk ‘{print “mv "”$1"" /mnt/example/Pictures"}’ > /tmp/movefiles.sh

        Then check the movefiles.sh and make sure it has all of the commands and files properly stated, make that executable, and then run that.

        • @mindbleachOP
          link
          15 months ago

          ls | grep | mv would work, except the StackOverflow discussion also highlights how parsing ls can have the same issues.

          I am moving thousands of files at once. If I have to check each one, it’s still wrong.

          The pragmatic answer turns out to be ./* instead of *.

          • @FigMcLargeHuge
            link
            1
            edit-2
            5 months ago

            I saw that answer and was just offering another option. I am sure xargs might work, but you would need to test as you need a destination passed on each line. Back to my way, I have used it for a lot more than just the move command. I think I used it to do a chmod once where I wanted to check and make sure before I committed to actually running the command(s). You could also use find and the -exec option, which I think was also mentioned here.

            Edit: also, you wouldn’t need to check each one, just the first few and last few to make sure the syntax is correct. Maybe do a wc -l to make sure it’s got the right number of entries and then let it run.

    • @[email protected]
      link
      fedilink
      35 months ago

      I’m also a bit lost. From the final sentence “having it put each name in quotes should be fucking trivial.” OP seems to complain that filenames with spaces expand into multiple arguments. I can’t recreate this with this command in bash:

      $ printf %s\\n *.jpg
      

      This command prints each argument in its own line. Replace *.jpg with whatever to prove it to yourself. Filenames with spaces show in full in a single line for me.

    • @mindbleachOP
      link
      15 months ago

      It expands *.jpg into a list of names.

      Which you’d think would mean mv "thing.jpg" "other thing.jpg", but no, it does mv thing.jpg other thing.jpg - and then chokes, because other is not a valid option or filename. Possibly after writing the contents of other thing.jpg onto the destination thing.jpg.

      If any filenames contain characters like ; or &, you better hope the next word doesn’t look like a command.

      • @[email protected]
        link
        fedilink
        55 months ago

        Which shell are you using? This does not happen in Bash, Zsh, or any modern shell I’m familiar with.

      • @[email protected]
        link
        fedilink
        1
        edit-2
        5 months ago

        What shell are you using? Is this exclusive to mv (and maybe other builtins)?

        Edit: also why would you ever want to move files like this, what is wrong with you

        • @mindbleachOP
          link
          -25 months ago

          Edit: also why would you ever want to move files like this, what is wrong with you

          … like what?

          The fuck do you mean, why would I want to move files in a terminal? Because I’m doing shit in a terminal! What, do you expect people to Super+E, find the directory they’re in, and drag-and-drop? I’m using tools that only exist in the command line.

          I’m using whatever GNOME Terminal came with Linux Mint 20 by default. StackOverflow answers sound like any POSIX terminal will do this. I assume it’s standard behavior.

          • @[email protected]
            link
            fedilink
            25 months ago

            I’m using whatever GNOME Terminal came with Linux Mint 20 by default.

            Should be bash. Type help it should tell you.

            StackOverflow answers sound like any POSIX terminal will do this. I assume it’s standard behavior.

            Link?

          • @[email protected]
            link
            fedilink
            1
            edit-2
            5 months ago

            I mean with * where you have no control of the order of the files and you never explicitly say which files are being moved. “yeah, just rename files into other files, I don’t fucking care lol” is what your command does