Not possible to run in docker? "critical: unshare failed (1): Operation not permitted"

[ostrolucky]

❯ cat Dockerfile
FROM debian:10

RUN apt-get update
RUN apt-get install -y curl

RUN curl -Ss https://api.github.com/repos/tycho-kirchner/shournal/releases/latest | grep "browser_download_url.*deb" | cut -d : -f 2,3 | tr -d \" | xargs curl -LSso shournal.deb
RUN apt install -y ./shournal.deb && rm shournal.deb

#RUN curl -LSs https://raw.githubusercontent.com/tycho-kirchner/shournal/master/shell-integration-scripts/bash/bash_integration.sh > /
#RUN SHOURNAL_ENABLE

RUN echo 'export HISTSIZE=2; source /usr/share/shournal/SOURCE_ME.bash && SHOURNAL_ENABLE' >> /root/.bashrc
❯ docker build .
WARN[0000] invalid non-bool value for BUILDX_NO_DEFAULT_LOAD:
[+] Building 0.0s (10/10) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                              0.0s
 => => transferring dockerfile: 32B                                                                                                                                                                               0.0s
 => [internal] load .dockerignore                                                                                                                                                                                 0.0s
 => => transferring context: 2B                                                                                                                                                                                   0.0s
 => [internal] load metadata for docker.io/library/debian:10                                                                                                                                                      0.0s
 => [1/6] FROM docker.io/library/debian:10                                                                                                                                                                        0.0s
 => CACHED [2/6] RUN apt-get update                                                                                                                                                                               0.0s
 => CACHED [3/6] RUN apt-get install -y curl                                                                                                                                                                      0.0s
 => CACHED [4/6] RUN curl -Ss https://api.github.com/repos/tycho-kirchner/shournal/releases/latest | grep "browser_download_url.*deb" | cut -d : -f 2,3 | tr -d " | xargs curl -LSso shournal.deb                 0.0s
 => CACHED [5/6] RUN apt install -y ./shournal.deb && rm shournal.deb                                                                                                                                             0.0s
 => CACHED [6/6] RUN echo 'export HISTSIZE=2; source /usr/share/shournal/SOURCE_ME.bash && SHOURNAL_ENABLE' >> /root/.bashrc                                                                                      0.0s
 => exporting to image                                                                                                                                                                                            0.0s
 => => exporting layers                                                                                                                                                                                           0.0s
 => => writing image sha256:60a0a1f510a47953157e73e9348796e5d36d62d5ff5c3fe54c14d4ac367f95f3
❯ docker run --rm -it 60a0a1f510a47953157e73e9348796e5d36d62d5ff5c3fe54c14d4ac367f95f3 bash
shournal-run 2021-03-30 21:12:39 critical: unshare failed (1): Operation not permitted
shournal shell-integration 2021-03-30 21:12:39 critical: Setup of external shournal-run-process failed, received message: Bad response (-1)
root@40110ef519d3:/# touch foo
shournal-run 2021-03-30 21:12:42 critical: unshare failed (1): Operation not permitted
shournal shell-integration 2021-03-30 21:12:42 critical: Setup of external shournal-run-process failed, received message: Bad response (-1)
root@40110ef519d3:/#

[tycho-kirchner]

It is possible to run shournal inside docker, thanks for this reminder for me to update the README. Does the procedure in issue 2 solve it for you?
Btw, in a few days I hope to release the next version with a new backend, which will not require these capabilites, you might want to take a look (;

[ostrolucky]

Indeed that solves the error. But not sure it works correctly:

❯ docker run --rm -it --cap-add SYS_ADMIN --cap-add SYS_PTRACE --cap-add SYS_NICE 60a0a1f510a47953157e73e9348796e5d36d62d5ff5c3fe54c14d4ac367f95f3 bash
root@b2964f075422:/# echo foo > bar
root@b2964f075422:/# shournal --query --wfile bar
No results found matching the query.
root@b2964f075422:/# touch baz
root@b2964f075422:/# shournal --query --wfile baz
cmd-id 3: $?: 0 31 Mar 2021 15:41:49 - 31 Mar 2021 15:41:49 :  touch baz
Working directory: /
session-uuid j1/cHpI3EeuHVAJCrBEAAg==
  1 written file(s):
     //baz (0 bytes) Hash: -
root@b2964f075422:/# echo foo >> baz
root@b2964f075422:/# shournal --query --wfile baz
No results found matching the query.
root@b2964f075422:/# ls
bar  baz  bin  boot  dev  etc  home  lib  lib64  media	mnt  opt  proc	root  run  sbin  srv  sys  tmp	usr  var

edit: Using a MacOS here as host system

[tycho-kirchner]

Thanks for spotting and reporting this bug, which affects the shell-integration but not when executed via
root@481d198040b5:/# shournal -e sh -c 'echo foo > bar'. Further, launched external commands (./script.sh) are not affected.
It occurs, if the working-directory is / and the opened filepath is relative ( echo foo > /bar works) or
for malformed file-paths with multiple leading slashes ////bar. So cd /tmp && echo foo > bar works.
For now I recommend to perform your work in any other working directory but the root-dir (;
It will be fixed in the next release. In case you want to compile from source:

src/shell-integration/event_open.cpp

@@ -43,7 +43,9 @@ static std::string mkAbsPath(const char* path){
 
     // resize to actual length
     buf.resize(strlen(rawBuf));
-    buf += '/';
+    if(buf.size() != 1){
+        buf += '/';
+    }
     buf += path;
     return buf;
 }
@@ -69,9 +71,21 @@ int event_open::handleOpen(const char *pathname, int flags, mode_t mode, bool la
     if(g_shell.watchState != E_WatchState::WITHIN_CMD){
         return g_shell.orig_open(pathname, flags, mode);
     }
-
     const auto absPath = mkAbsPath(pathname);
-    if(absPath.size() < 2){
+
+    // pass the resolved abs. path relative to shournal's root directory fd,
+    // by omitting the initial '/'.
+    // Users may further pass malformed file-paths such as //foo, so find the first
+    // non-slash char.
+    const char* actualPath = nullptr;
+    for(size_t i=0; i < absPath.size(); i++){
+        if(absPath[i] != '/'){
+            actualPath = &absPath[i];
+            break;
+        }
+    }
+
+    if(actualPath == nullptr || absPath.c_str() + absPath.size() - actualPath < 1){
         // Get here on mkAbsPath-error or because user attempted to open "/" or ""
         // The shortest possible absolute FILEpath under linux is two chars long.
         // We may get here, if bash-user calls e.g.
@@ -79,8 +93,8 @@ int event_open::handleOpen(const char *pathname, int flags, mode_t mode, bool la
         logDebug << "no valid path" << absPath;
         return g_shell.orig_open(pathname, flags, mode);
     }
-    // pass the resolved abs. path relative to shournal's root directory fd,
-    // by omitting the initial '/'.
-    return openat(g_shell.shournalRootDirFd, absPath.c_str() + 1, flags, mode);
+
+    logDebug << "about to open" << actualPath - 1;
+    return openat(g_shell.shournalRootDirFd, actualPath, flags, mode);
 }

Also added a test-case for the shell-integration:

test/integration_test_shell.cpp

     void testWrite() {
         auto pTmpDir = testhelper::mkAutoDelTmpDir();
         auto tmpDirPath = pTmpDir->path().toStdString();
-
+        QVERIFY(tmpDirPath != "/"); // otherwise this test must be changed
+        auto tmpDirNoLeadingSlash(tmpDirPath);
+        tmpDirNoLeadingSlash.erase(tmpDirNoLeadingSlash.begin());
 
         std::string filepath = tmpDirPath + "/f1";
         std::vector<std::string> cmds {
@@ -150,26 +169,37 @@ private slots:
                     "(echo foo8 > " + filepath + ") & wait",
                     "/bin/echo foo9 > " + filepath + " & wait",
                     "sh -c 'echo foo10 > " + filepath + " & wait'",
+                    // malformed filepath with multiple slash //
+                    "echo foo11 > //" + filepath,
+                    // special case root dir
+                    "cd /; echo foo11 > //" + filepath,
                     // relative paths must also work:
                     "cd " + tmpDirPath + "; echo hi > f1",
                     "cd " + tmpDirPath + "; echo hi > ./f1",
                     "cd " + tmpDirPath + "; echo hi > ../" + splitAbsPath(tmpDirPath).second + "/f1",
+                    // special case root dir
+                    "cd /; echo hi > " + tmpDirNoLeadingSlash + "/f1",
         };

[ostrolucky]

Cool. I'll leave the fate of this issue up to you then. Close it or keep it as placeholder until those special args for docker are not needed.

[tycho-kirchner]

Indeed that solves the error. But not sure it works correctly:

❯ docker run --rm -it --cap-add SYS_ADMIN --cap-add SYS_PTRACE --cap-add SYS_NICE 60a0a1f510a47953157e73e9348796e5d36d62d5ff5c3fe54c14d4ac367f95f3 bash
root@b2964f075422:/# echo foo > bar
root@b2964f075422:/# shournal --query --wfile bar
No results found matching the query.
root@b2964f075422:/# touch baz
root@b2964f075422:/# shournal --query --wfile baz
cmd-id 3: $?: 0 31 Mar 2021 15:41:49 - 31 Mar 2021 15:41:49 :  touch baz
Working directory: /
session-uuid j1/cHpI3EeuHVAJCrBEAAg==
  1 written file(s):
     //baz (0 bytes) Hash: -
root@b2964f075422:/# echo foo >> baz
root@b2964f075422:/# shournal --query --wfile baz
No results found matching the query.
root@b2964f075422:/# ls
bar  baz  bin  boot  dev  etc  home  lib  lib64  media	mnt  opt  proc	root  run  sbin  srv  sys  tmp	usr  var

edit: Using a MacOS here as host system

Fixed as of be900ad

And the actual issue (running shournal inside docker) is as of v2.4 documented in the general README - for both backends, fanotify and kernel module. Please feel free to provide feedback, if there is a further need for clarification.