[bash] String manipulation within

In stead of turning to subprocesses with sed and the like. Works at least as of version 4.2.46(2)

# _LAST_PATCH being something like 00.0042. It is a real
# world example and as such not everything makes sense. _MAJOR for instance
# is just dead weight.
shopt -s extglob                    # To get support for more sophisticated pattern
_MAJOR=${_LAST_PATCH%%.+([0-9])}    # Remove longest pattern match from behind
_MINOR=${_LAST_PATCH##+([0-9]).}    # Remove longest pattern match from front
_MINOR=${_MINOR##+(0)}              # Remove leading 0s
_MAJOR=${_MAJOR:-0}                 # If nothing remains, set to 0
_MINOR=${_MINOR:-0}
shopt -u extglob

# Generate the new minor number
((_MINOR += 1))                     # Increment _MINOR
_MINOR=$(printf %04d ${_MINOR})     # left padding with 0. printf is a bash internal!

[Gnu tools] find quirks

Intended effect

List all directories recursively called "ready" or "data" .

Failing command

find . -type d -a \(-name ready -o -name data\)

find: paths must precede expression: `(-name' 

Cause

find is somewhat picky with respect to parenthesis. It is not enough to escape them.

Solution

Make sure, whitespace is around them.

find . -type d -a \( -name ready -o -name data \)

[Video] for Bluray-Player Samsung

Supported formats

HT-J4500

Tested

Result	Container format	Video Codec	Audio Codec	Remarks
OK	raw	raw	raw	I noticed that at least some DVDs can be copied to a NAS disk which in turn is connected and read from the player just as they appear on the file system browser under the Windows 10 explorer/FileZilla. I assume it does not matter whether this is done with the via of the local file system as I did or directly or whether it is important to do it under Windows. It was an accidental discovery. Unfortunately and obviously, copy protected media deny such treatment - at least under Windows 10.
OK	Matroska (.mkv)	Advance Video Coding (AVC/h.264/MPEG-4 Part 10)	MPEG 3 (MP3)
OK	MPEG 4 (.3gp / .3gpp)	MPEG 4 (Advanced Simple BVOP)	Advanced Audio Coding (AAC)
OK	QTFF Quicktime file format (.mov)	Advance Video Coding (AVC/h.264/MPEG-4 Part 10)	Pulse Code Modulation (PCM)
OK	MPEG PS (.vob)	MPEG 2 (Main BVOP)	MPEG 2 (MP2)	The file suffix of the container format seems to be wrong
NOK	Matroska (.mkv)	MPEG 2 (MP2)	Pulse Code Modulation (PCM)	Unsupported format error (it is assumed that mkv is meant even tough specs tells it was (possibly in combination with the video codec)
NOK	Matroska (.mkv)	MPEG 2 (MP2)	Dolby Digital / AC-3	Unsupported format error (it is assumed that mkv is meant even tough specs tells it was (possibly in combination with the video codec)
Result	Container format	Video Codec	Audio Codec	Remarks

avi

• Video
  
  • DivX 3.11/4/5/6
  • mp4 v3
  • H.264 BP/MP/HP

•  Audio
• mp3
• WMA
• PCM
• Dolby Digital (= AC-3)
• DTS

mkv

• Video
• VC-1 AP
• DivX 5/6
• H.264 BP/MP/HP
• Audio
• mp3
• Dolby Digital (= AC-3)
• DTS
• Problems
• A conversion from H.264/opus to H.264/mp3 with copy or omitting -vcodec made first player complain about an unsupported format.

wmv (9)

• Video
• VC-1 AP/SM
• Audio
• WMA

mp4

• Video
• mp4v
• H.264 BP/MP/HP
• Audio
• AAC

BD-F5100

avi

• Video
  
  • DivX 3.11/4.x/5.x/6.1
  • mp4 v3
  • H.264 BP/MP/HP

•  Audio
• mp3
• WMA
• PCM
• Dolby Digital (= AC-3)
• DTS

mkv

• Video
• VC-1 AP
• DivX 5.1/6.0
• H.264 BP/MP/HP
• Audio
• mp3
• Dolby Digital (= AC-3)
• DTS

wmv (9)

• Video
• VC-1 AP/SM
• Audio
• WMA

mp4

• Video
• mp4v
• H.264 BP/MP/HP
• Audio
• AAC

Conversion with ffmpeg (3.4.4-0ubuntu0.18.04.1)

To convert audio/video use switches -acodec and -vcodec respectively. The value following the switch specifies the target format. If you do not want to convert one of the streams, you can omit the switch entirely (it seems to get very slow though) - or you use copy.
However, I had some troubles with my tests. It was a download from youtube where the video was mp4 stream and the audio webm (opus). youtube-dl complained "WARNING: Requested formats are incompatible for merge and will be merged into mkv." So I ended up with a Matroska container. This probably is because as of time of writing "opus in MP4 support is experimental, add '-strict -2' if you want to use it." as ffmpeg told me.

• Change audio to mp3
  

  ffmpeg -i some_file.mkv -acodec mp3 -vcodec copy some_file_of_other_container.mp4

• Change video to H.265
  

  ffmpeg -i some_file.mkv -vcodec hevc some_file_h265.mkv

[SQL] Efficient update with data from other table

Go along the following template:

• If you have columns to update from the same table set, use column set updates.
• Use correlated subqueries for calculation of the values as well as for restriction of the rows to update. There you can employ an exists clause

DB2 is said to be able to recognise that both subselects are identical and therefore won't resolved twice. I suppose any clever database works like this.

update TO_BE_UPDATED U
  set (COL_1, COL_2) = (
         select S.ATT_3, R.ATT_4
           from SOURCE_TABLE S
          inner join RESTRICTING_TABLE R
             on S.KEY_S = R.KEY_R
          where R.SOME_CONDITION between SOME_VALUE and SOME_OTHER_VALUE
            and S.KEY_S = U.KEY_U
      )
where exists (
         select S.ATT_3, R.ATT_4
           from SOURCE_TABLE S
          inner join RESTRICTING_TABLE R
             on S.KEY_S = R.KEY_R
          where R.SOME_CONDITION between SOME_VALUE and SOME_OTHER_VALUE
            and S.KEY_S = U.KEY_U
      )

Backdated changes in data history in DWH

Abbreviations

• DWH: Data Warehouse (probably trivial when you were interested in reading so far 😉)
• KF: Point in time from when the record is deemed known
• sKF: smaller VF, i. e. the VF of the record known next earlier to the reference (new) record
• bKF: bigger VF, i. e. the VF of the record known next later to the reference (new) record
• KT: Point in time when the record ceases to be valid
• RP: Reference point in time
• HP: High-point-in-time, i. e. an arbitrary point in time high enough not to get mixed up in operation, usually the maximum point in time a database can represent. It is a convenience replacement for SQL null value to ease querying.
• BK: Business key, i. e. not the technical key of this DWH but maybe the technical key of the delivering system or the real identifying key of the business entity. May be composed.
• DR: Delete record, i. e. a record delivered to make cease the validity of a record of the same BK existing in the target.
• IR: Insert record, i. e. a record delivered to create valid record (anew)
• CR: Change record, i. e. a record delivered to create a new version of a record of the same BK existing in the target.

Premises

• No parallel processing of data or alternatively processing of the data with repeatable read isolation to prevent lost updates.
• VT is meant exclusively, i. e. the valid-to boundary does not belong the interval defined by VF and VT.
• As backdated changes are permitted an CR or DR without existing valid

Directions

• Attribute sKF and bKF (e. g. with lead/lag functions) to the new record
• Distinguish the following cases:
• neither sKF nor bKF have been found
• DR: Into the target insert the new record with KF = RP and KT = RP.
• IR/CR: Into the target insert the new record with KF = RP and KT = HD.
• sKF but no bKF has been found
• DR: In the target update the record of BK with KF = sKF such that KT = RP.
• IR/CR:
• In the target update the record of BK with KF = sKF such that KT = RP.
• Into the target insert the new record with KF = RP and KT = HD.
• bKF but no sKF has been found
• DR: Into the target  insert the new record with KF = RP and KT = RP.
• IR/CR: Into the target  insert the new record with KF = RP and KT = bKF.
• sKF and bKF have been found
• DR: In the target update the record of BK with KF = sKF such that KT = RP.
• IR/CR:
• In the target update the record of BK with KF = sKF such that KT = RP.
• Into the target insert the new record with KF = RP and KT = bKF.

Trivia

I very much prefer the term "know" over "valid" as usually the business history of data is built by the primary system and delivered as a payload attribute (opposed to the key attribute(s)). The time axis that the DWH is to build when keeping the history of data is the one of when the characteristics of data of a specific business key gets known by the DWH and when this knowledge gets erased. E. g. at 2001-01-01 a customer registers ergo business validity is from 2001-01-01. This record gets processed and delivered to the DWH at 2001-01-02 ergo the known from is this point in time. The customer thinks twice and quits its business relation ship the next day after registration i. e. at 2001-01-02. This fact gets known by the DWH the next day again i. e. at 2001-01-03.
Delivery 1 at 2001-01-02 to the DWH:

OPERATION	VALID_FROM	VALID_TO	BK	EMAIL
Insert	2001-01-01	{null}	XYZ	foo@b.ar

DWH state at end of 2001-01-02:

KNOWN_FROM	KNOWN_TO	VALID_FROM	VALID_TO	BK	EMAIL
2001-01-02	{null}	2001-01-01	{null}	XYZ	foo@b.ar

Delivery 2 at 2001-01-03 to the DWH:

OPERATION	VALID_FROM	VALID_TO	BK	EMAIL
Delete	2001-01-01	2001-01-02	XYZ	foo@b.ar

DWH state at end of 2001-01-03:

KNOWN_FROM	KNOWN_TO	VALID_FROM	VALID_TO	BK	EMAIL
2001-01-02	2001-01-03	2001-01-01	{null}	XYZ	foo@b.ar
2001-01-03	{null}	2001-01-01	2001-01-02	XYZ	foo@b.ar

[Linux] Boot loop after new installation (on Toshiba Portege Z30-a)

UEFI boot stuff is some what sad story. It is said that M$ does not stick to its standard and "BIOS" manufacturers quite often prefere to do this entreprise's bidding. This can lead to problems with non-Windows OS to boot. See also https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface#Criticism. If I remember correctly, Windows overwrites fall back files in EFI/Boot eventhough it has its own directory as all OS are requested to use.
I have run into a problem related to this when Debian Stretch made a system upgrade. A fresh install of it or (l)ubuntu did not remedy.

Symptom

After a system installation or a system upgrade grub does not get loaded. After the manufacturer splash screen a message beginning with "System BootOrder not found." appears for a second or two and the computer reboots.

Cause

The wrong file is at EFI/Boot/bootx64.efi

Solution

Copy EFI/<your operating system>/grubx64.efi to afore mentioned lockation like

sudo mount /dev/sda1 /mnt
sudo cp /mnt/EFI/ubuntu/grubx64.efi /mnt/EFI/Boot/bootx64.efi

[Blogger] Integration of Alex Gorbatshev's syntax highlighter

To integrate Alex Gorbatshev's syntax highlighter you need to add following code to possibly the end of the head tag of the html design template.

<!-- Syntax Highlighter Additions START -->
<link href='http://alexgorbatchev.com/pub/sh/current/styles/shCore.css' rel='stylesheet' type='text/css'/>
<link href='http://alexgorbatchev.com/pub/sh/current/styles/shThemeEmacs.css' rel='stylesheet' type='text/css'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shCore.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushAS3.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushBash.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushColdFusion.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCSharp.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCpp.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCss.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushDelphi.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushDiff.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushErlang.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushGroovy.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushJScript.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushJava.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushJavaFX.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPerl.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPhp.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPlain.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPowerShell.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPython.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushRuby.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushScala.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushSql.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushVb.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushXml.js' type='text/javascript'/>
<script language='javascript' type='text/javascript'>  SyntaxHighlighter.config.bloggerMode = true;  SyntaxHighlighter.all(); </script>
<!-- Syntax Highlighter Additions END -->

You can get there as follows.

For its usage see also using-my-configuration-of-alex