Shell Store variable string with single quote AND double quote AND asterisk

Perceval

Perceval

Hello !
Kind of unexpected nightmare here. I run Dolibarr for my business, which is mainly a PHP website. I had to change some pieces of code for my own usage, and I'm shell-scripting (NO bash ! sh only) everything fo convenience (my script updates my Dolibarr to its latest version, and now I added in my script the pieces of code that it will update automatically since they'll be overwritten).
BUT... One line I have to change is very funny to handle. Here it is :
Code: 
dol_concatdesc($libelleproduitservice, " * ".$subprodval[5].(($subprodval[5] && $subprodval[3]) ? ' - ' : '').$subprodval[3]),

That's the original source line, and this is the one I'll get after updating my setup. There are both double quotes, single quotes and the asterisk. I need to search EXACTLY this line (using sed) to make sure only this line will be edited. I can't base my edition on line number as it is expected there would be lines added or removed by devs. I'm storing this whole string into a variable I send to sed ( sed -i '.orig' -e 's!toriginal!replaced!g' $file).

I've been able to get most of the line by splitting the variable in two parts, playing ugly with single and double quotes :
Code: 
PDFLIB_L1674_1='dol_concatdesc($libelleproduitservice, " * "'
PDFLIB_L1674_2=".$subprodval[5].(($subprodval[5] && $subprodval[3]) ? ' - ' : '').$subprodval[3]),"

It seems to work, but the asterisk will not. My shell script thinks I'm passing *, so it prints inside the variable the list of my current directory. This input variable is ugly, as I need to keep everything exactly as the original but I set my output variable with single quotes only (I don't know why the devs mixed them) and I can change the * with anything I want (it's used to print kind-of item point on the invoices).

Does anyone know a nice and clean way to fill this string into a variable, and make understand the script to keep the content as a string ?

Feel free to tell me if I'm doing it the wrong way, I don't pretend my method to be the good one. Thanks very much !!
 
Code: 
PDFLIB_L1674_1="dol_concatdesc(\$libelleproduitservice, \" \* \""

Edit: no, sorry it doesn't work. It prints \* instead of *.

After somes tries, I think the dir list is displayed when you print the var on the screen. The var keeps its * untouched, in fact. In a sed filter you may need to add a \ before if the substitution doesn't work.
 
Thanks for trying ! Well, I don't know why, but today it works... ?!
cracauer@ : I just tried, and it works too. I'm keeping your version because it's safer, I just don't understand why it looks fine today. Did something change since 14.3->15.0...?
I forgot to say that I'm protecting the variable using double quotes when I play with it. IE : echo "$PDFLIB_L1674_1$PDFLIB_L1674_2"
 
Perceval said:
Thanks for trying ! Well, I don't know why, but today it works... ?!
cracauer@ : I just tried, and it works too. I'm keeping your version because it's safer, I just d'ont understand why it looks fine today. Did something change since 14.3->15.0...?
Click to expand...

I didn't follow sh changes lately, so I woudln't know.

You can always cross-check with bash.
 
I'll keep you updated, I need to check further, as my shell thinks the PHP variables of my edited lines are for it... 😂
 
Perceval said:
[...]
Code: 
dol_concatdesc($libelleproduitservice, " * ".$subprodval[5].(($subprodval[5] && $subprodval[3]) ? ' - ' : '').$subprodval[3]),
[...]
Does anyone know a nice and clean way to fill this string into a variable, and make understand the script to keep the content as a string ?

Feel free to tell me if I'm doing it the wrong way, I don't pretend my method to be the good one. Thanks very much !!
Click to expand...
I know nothing about Dolibarr or the rest of your environment.
I may be able to say something about sed.

I'm not clear about what you have exactly and what you need.
dol_concatdesc($libelleproduitservice, " * ".$subprodval[5].(($subprodval[5] && $subprodval[3]) ? ' - ' : '').$subprodval[3]),
Is this the exact and complete string we're talking about?
What are its unique qualities, e.g. is the call dol_concatdesc( unique and does it allways come at the beginning of a line?
Does this string occur only once in a line and can these substitutions be made on a line-by-line basis? If you have the source completely on one line, that changes everything.

Start with sed in the non-inline way; that way one can have a better understanding of the effects of the sed command.
What exactly is the contents of
toriginal
replaced

Furthermore, if these represent sh variables, the usual way to go about it with sed would be:
Code: 
sed -i '.orig' -e "s!${toriginal}!${replaced}!g" $file
You cannot evaluate shell/environment variables inside strong quotes (i.e. single quotes); for that you must use double quotes.

However, there may be all sorts of hidden problems in the source and target strings.
Also what's in
$file
 
Thanks everybody ! I do enjoy playing with scripts but my level is very low.
covacat well... I don't really know how these commands work, so I stayed in my "comfort area" with the commands I know. But as I said, I might go wrong using sed ! Do they allow backups ? Using sed makes a backup on a single line, that's nice.
Erichans sorry if I wasnt clear.
I need to change -I'd rather say append, that's in fact what I'm doing the dirty way by replacing the whole line instead of appending. I need on other strings to change for instance the last character (ie a coma) of the original file. I do know I can do better but for now I'm trying this way...
The afformentionned string is indeed the exact and complete one. That's the one I'll get after updating, replacing my edited lines by a clean copy. The line content is indeed unique, where components are not (ie dol_concatdesc( is called several times inside the PHP file) and as stated, the line number is not relevant, as any change made on the file by the devs would move the things. That's why I'm basing my sed search on it. Indeed, the sed command is not ready yet, as I was trying first to get the variables $toriginal and $replaced correct (won't probably their final names, I just put these as a reminder). The "!" are here to get a unique separator that I'm not supposed to find in these strings.
$file is the target file to edit, here for instance (several files in fact) : /dolibarr/product/class/product.class.php
These variables are perfectly stored and work nicely (ie if I run test on it).
 
Perceval said:
The "!" are here to get a unique separator that I'm not supposed to find in these strings.
Click to expand...
Yes that's how to go about it.

Code: 
sed -i '.orig' -e "s!${toriginal}!${replaced}!g" $file
This, that is the g modifier, suggests that the targetsource string may occur multiple times on a line.
Is that in fact the case?

Is the replacement string itself constant, to be more precise: are the characters in the replacement string constant?

I really need the replacement string; there may very well be extra consirations there as well.
 
well diff and patch are kinda made for such stuff
and yes you have a backup by default

the ports system uses this method against the original/upstream distribution of the ported software to create the 'freebsd version' out of it
the patches do not change as often as the distribution changes
the patch file contains the context of the changes so even if the specific line moves up or down it still finds it
 
Perceval said:
[…] I've been able to get most of the line by splitting the variable in two parts, playing ugly with single and double quotes : […]
Click to expand...
You can store everything in one single variable by using a non‑interpolated here‑document, the terminator string in single quotes disables interpolation.​
Bash: 
LOC=$(cat << 'EOT')
dol_concatdesc($libelleproduitservice, " * ".$subprodval[5].(($subprodval[5] && $subprodval[3]) ? ' - ' : '').$subprodval[3]),
EOT
Perceval said:
[…] I need to search EXACTLY this line (using sed) to make sure only this line will be edited. I can't base my edition on line number as it is expected there would be lines added or removed by devs. I'm storing this whole string into a variable I send to sed ( sed -i '.orig' -e 's!toriginal!replaced!g' $file). […]
Click to expand...
The stupid thing is that you can’t disable interpretation of metacharacters in sed(1). covacat already mentioned the proper approach using a patch(1). If you really wanted to concoct your own thing:​
Bash: 
{
	# The offending LOC.
	cat <<- 'EOT'
		dol_concatdesc($libelleproduitservice, " * ".$subprodval[5].(($subprodval[5] && $subprodval[3]) ? ' - ' : '').$subprodval[3]),
		EOT
} | fgrep -n -f - "${funky_file}" | while IFS=':' read -r line_number match
do
	{
		# Line address. Change line.
		printf '%i%s\n' "${line_number}" 'c'
		# The line’s new contents.
		cat <<- 'EOT'
			dol_concatdesc($libelleproductservice, " * ".$subprodval[5].(($subprodval[5] && $subprodval[3]) ? ' - ' : '').$subprodval[3]),
			EOT
		# Terminate input mode.
		printf '.\n'
		# Write and quit.
		printf 'wq\n'
	} | ed -s "${funky_file}"
done
 
Kai Burghardt wow ! That's impressive. I'm going to test this. I had a look on diff and patch and it's still a bit obsure to me, I think I'll have to dig this ways since it is the better way. I remember I used it on several ports, without really taking time to understand the underlying "how it works".
Thanks very much, I do appreciate. That's how we learn !! (Thanks a lot to Dolibarr too, but it's cynical : if what I wanted was supported or not eaten by outrageously expensive addons, all this would be useless...)
 
Perceval said:
Does anyone know a nice and clean way to fill this string into a variable,
Click to expand...
Probably not a clean way ...
If, however, given a literal replacement string, it looks like it can be done.

So far, with gsed, I got a targeted matched replacement (round trip) working; looks like it can be done.
Don't know if gsed is acceptable. Otherwise you may prefer to proceed with either a here-document or patch(1).

The problem is the single quote in sed*:
Code: 
[1-0] % echo "a'c" | sed 's/a'c/abc/'
Unmatched '''.
[2->1<] % echo "a'c" | gsed 's/a\o047c/abc/'
abc

I really need the specifics about that replacement string if exploring further with gsed.

___
* Edit: you can make matching with sed less strict: matching ' with any character .:
Code: 
$ printf 'a\047c\$e\n' | sed -E 's!a.c\$e!abcd!'
abcd

But hey, sed(1) isn't really for patch-work ;)
 
covacat said:
then you need to escape all the frickin $ [] . and whatnot from the regex
Click to expand...
Yes: "not a clean way ..."

That's why I asked about the specifics of the replacement string.

So far, X marks the spot:
Rich (BB code): 
[1-0] % cat s2.sh
#!/bin/sh
gsed -E -e \
'\!dol_concatdesc\(\$libelleproduitservice, " \* "\.\$subprodval\[5\]\.\(\(\$subprodval\[5\] \&\& \$subprodval\[3\]\) \? \o047 - \o047 : \o047\o047\)\.\$subprodval\[3\]\),!  s!dol_concatdesc\(\$libelleproduitservice, " \* "\.\$subprodval\[5\]\.\(\(\$subprodval\[5\] \&\& \$subprodval\[3\]\) \? \o047 - \o047 : \o047\o047\)\.\$subprodval\[3\]\),!X!' \
t1.in
[2-0] % cat t1.in
first line
dol_concatdesc($libelleproduitservice, " * ".$subprodval[5].(($subprodval[5] && $subprodval[3]) ? ' - ' : '').$subprodval[3]),
last line
[3-0] % ./s2.sh
first line
X
last line
[4-0] %
 
Kai Burghardt said:
LOC=$(cat << 'EOT')
dol_concatdesc($libelleproduitservice, " * ".$subprodval[5].(($subprodval[5] && $subprodval[3]) ? ' - ' : '').$subprodval[3]),
EOT
Click to expand...

Very similar idea; read can also be of use here (and my brain finds it easier to parse):
Bash: 
read LOC <<'EOT'
dol_concatdesc($libelleproduitservice, " * ".$subprodval[5].(($subprodval[5] && $subprodval[3]) ? ' - ' : '').$subprodval[3]),
EOT
 
Hi !
Well, Kai Burghardt version using EOT seems to do the trick. It does keep both single quotes, double quotes and the asterisk. It does also protect the PHP variables from being wrongly used by shell. It is easier than my method of splitting variables, which is always a possible source of human errors.
I had no time to go more, but sed does not seem to work correctly. I keep you updated !
 
Erichans said:
Kai Burghardt said:
The stupid thing is that you can’t disable interpretation of metacharacters in sed(1).​
Click to expand...
What do you mean?
Click to expand...
I meant something akin to grep(1)’s ‑‑fixed‑strings. Escaping works, yeah, but you want to keep your source code reasonably easily verifiable as well (KISS).​
Eric A. Borisch said:
Very similar idea; read can also be of use here (and my brain finds it easier to parse): […]
Click to expand...
Right, yes, thank you, a single one‑line variable can be filled with the built‑in read. Multiple lines requires cat(1). Of course you meant read -r, the ‑r disabling interpretation of backslash escaping – for now there aren’t any backslashes, but it’s really easy to forget that detail should the LOC change to contain backslashes.​
 
You must log in or register to reply here.
Back
Top