[Fawkes Git] branch/fzwilling/robot-memory: 8 revs updated. (0.5.0-3405-g5173577)

Frederik Zwilling zwilling at kbsg.rwth-aachen.de
Wed Sep 28 18:44:11 CEST 2016


Changes have been pushed for the project "Fawkes Robotics Software Framework".

Gitweb: http://git.fawkesrobotics.org/fawkes.git
Trac:   http://trac.fawkesrobotics.org

The branch, fzwilling/robot-memory has been updated
        to  5173577ed6812e699e22143881e221a09629e707 (commit)
       via  19d0fbfafa5901603ac07a8dd13cc6021f0350a0 (commit)
       via  d5cc3a8454a34cb3661f409dcfd67262fb5f5b0d (commit)
       via  365c95966cad4aac69ea73323cec4f2098ab1531 (commit)
       via  0ba989a955956994feb728db46859943183c089f (commit)
       via  291d8475b466d01f1be661bcc8a5b1a96f923ef4 (commit)
       via  a03d31e17ffb0d8a1d5ddb1a791c5d5f4dddab8b (commit)
       via  67666871ebc5ba66f3b874a6448dfdac84d52435 (commit)
      from  ae75a1bf012916533f7c608c5d61ed8b53e706c0 (commit)

http://git.fawkesrobotics.org/fawkes.git/fzwilling/robot-memory

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- *Log* ---------------------------------------------------------------
commit 67666871ebc5ba66f3b874a6448dfdac84d52435
Author:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
AuthorDate: Thu Sep 22 20:01:32 2016 +0200
Commit:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
CommitDate: Wed Sep 28 18:42:24 2016 +0200

    robot-memory: setup replicaset and mongos process

http://git.fawkesrobotics.org/fawkes.git/commit/6766687
http://trac.fawkesrobotics.org/changeset/6766687

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
commit a03d31e17ffb0d8a1d5ddb1a791c5d5f4dddab8b
Author:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
AuthorDate: Fri Sep 23 19:16:30 2016 +0200
Commit:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
CommitDate: Wed Sep 28 18:42:28 2016 +0200

    clips-robot-memory: add remove function

http://git.fawkesrobotics.org/fawkes.git/commit/a03d31e
http://trac.fawkesrobotics.org/changeset/a03d31e

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
commit 291d8475b466d01f1be661bcc8a5b1a96f923ef4
Author:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
AuthorDate: Wed Sep 28 13:06:57 2016 +0200
Commit:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
CommitDate: Wed Sep 28 18:42:32 2016 +0200

    robot-memory: config sharded cluster automatically

http://git.fawkesrobotics.org/fawkes.git/commit/291d847
http://trac.fawkesrobotics.org/changeset/291d847

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
commit 0ba989a955956994feb728db46859943183c089f
Author:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
AuthorDate: Wed Sep 28 14:51:15 2016 +0200
Commit:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
CommitDate: Wed Sep 28 18:42:36 2016 +0200

    robot-memory: set read preference 'nearest' to read locally

http://git.fawkesrobotics.org/fawkes.git/commit/0ba989a
http://trac.fawkesrobotics.org/changeset/0ba989a

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
commit 365c95966cad4aac69ea73323cec4f2098ab1531
Author:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
AuthorDate: Wed Sep 28 14:53:39 2016 +0200
Commit:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
CommitDate: Wed Sep 28 18:42:42 2016 +0200

    robot-memory-test: use simulated replica set in tests

http://git.fawkesrobotics.org/fawkes.git/commit/365c959
http://trac.fawkesrobotics.org/changeset/365c959

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
commit d5cc3a8454a34cb3661f409dcfd67262fb5f5b0d
Author:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
AuthorDate: Wed Sep 28 15:47:38 2016 +0200
Commit:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
CommitDate: Wed Sep 28 18:42:46 2016 +0200

    robot-memory: made setup robust against missing db+shards already added

http://git.fawkesrobotics.org/fawkes.git/commit/d5cc3a8
http://trac.fawkesrobotics.org/changeset/d5cc3a8

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
commit 19d0fbfafa5901603ac07a8dd13cc6021f0350a0
Author:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
AuthorDate: Wed Sep 28 16:18:59 2016 +0200
Commit:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
CommitDate: Wed Sep 28 18:42:53 2016 +0200

    robot-memory: use default db name when modified for simulation

http://git.fawkesrobotics.org/fawkes.git/commit/19d0fbf
http://trac.fawkesrobotics.org/changeset/19d0fbf

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
commit 5173577ed6812e699e22143881e221a09629e707
Author:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
AuthorDate: Wed Sep 28 18:35:30 2016 +0200
Commit:     Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
CommitDate: Wed Sep 28 18:44:03 2016 +0200

    robot-memory: added dependencies in plugins/Makefile

http://git.fawkesrobotics.org/fawkes.git/commit/5173577
http://trac.fawkesrobotics.org/changeset/5173577

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


- *Summary* -----------------------------------------------------------
 src/plugins/Makefile                               |    3 +-
 .../clips_robot_memory_thread.cpp                  |   15 +++-
 .../clips-robot-memory/clips_robot_memory_thread.h |    1 +
 src/plugins/robot-memory/robot_memory.cpp          |   36 +++++---
 src/plugins/robot-memory/robot_memory.h            |    2 +
 src/plugins/robot-memory/robot_memory_setup.cpp    |  104 +++++++++++++++-----
 src/plugins/robot-memory/robot_memory_setup.h      |    3 +
 src/tools/gtest/gtest_fawkes.cpp                   |    8 +-
 8 files changed, 132 insertions(+), 40 deletions(-)


- *Diffs* -------------------------------------------------------------

- *commit* 67666871ebc5ba66f3b874a6448dfdac84d52435 - - - - - - - - - -
Author:  Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
Date:    Thu Sep 22 20:01:32 2016 +0200
Subject: robot-memory: setup replicaset and mongos process

 src/plugins/robot-memory/robot_memory_setup.cpp |   69 ++++++++++++++---------
 src/plugins/robot-memory/robot_memory_setup.h   |    1 +
 2 files changed, 44 insertions(+), 26 deletions(-)

_Diff for modified files_:
diff --git a/src/plugins/robot-memory/robot_memory_setup.cpp b/src/plugins/robot-memory/robot_memory_setup.cpp
index 49fffb1..1b78571 100644
--- a/src/plugins/robot-memory/robot_memory_setup.cpp
+++ b/src/plugins/robot-memory/robot_memory_setup.cpp
@@ -45,37 +45,54 @@ RobotMemorySetup::~RobotMemorySetup()
 void RobotMemorySetup::setup_mongods()
 {
   //start local mongod if necessary
-  unsigned int local_port = config->get_int("plugins/robot-memory/setup/local/port");
-  if (!is_mongo_running(local_port))
-  {
-    const char *argv[] = {"mongod","--port", std::to_string(local_port).c_str(), NULL};
-    std::string cmd = command_args_tostring(argv);
-    logger->log_info("RobotMemorySetup", "Starting local mongod process: '%s'", cmd.c_str());
-    local_mongod = new SubProcess("mongod-local", "mongod", argv, NULL, logger);
-    logger->log_info("RobotMemorySetup", "Started local mongod");
-    wait_until_started(local_port, cmd);
-  }
+  unsigned int local_port = config->get_uint("plugins/robot-memory/setup/local/port");
+  std::string local_port_str = std::to_string(local_port);
+  const char *local_argv[] = {"mongod", "--port", local_port_str.c_str(), NULL};
+  start_mongo_process("mongod-local", local_port, local_argv);
 
   //only start other processes when we want to run the robot memory distributed
   if(!config->get_bool("plugins/robot-memory/setup/distributed"))
     return;
 
-  unsigned int config_port = config->get_int("plugins/robot-memory/setup/config/port");
-  if (!is_mongo_running(config_port))
-  {
-    std::string repl_set_name = config->get_string("plugins/robot-memory/setup/config/replica-set-name");
-    std::string db_path = StringConversions::resolve_path(config->get_string("plugins/robot-memory/setup/config/db-path").c_str());
-    const char *argv[] = {"mongod", "--configsvr", "--port", std::to_string(config_port).c_str(),
-        //"--replSet", repl_set_name.c_str(),
-        "--dbpath", db_path.c_str(), NULL};
-    logger->log_info("RobotMemorySetup", "Running on port: %s", std::to_string(config_port));
-    std::string cmd = command_args_tostring(argv);
-    prepare_mongo_db_path(db_path);
-    logger->log_info("RobotMemorySetup", "Starting config mongod process: '%s'", cmd.c_str());
-    config_mongod = new SubProcess("mongod-config", "mongod", argv, NULL, logger);
-    logger->log_info("RobotMemorySetup", "Started config mongod");
-    wait_until_started(config_port, cmd);
-  }
+  unsigned int config_port = config->get_uint("plugins/robot-memory/setup/config/port");
+  std::string db_path = StringConversions::resolve_path(config->get_string("plugins/robot-memory/setup/config/db-path").c_str());
+  prepare_mongo_db_path(db_path);
+  std::string config_port_str = std::to_string(config_port);
+  const char *config_argv[] = {"mongod", "--configsvr", "--port",
+      config_port_str.c_str(), "--dbpath", db_path.c_str(), NULL};
+  start_mongo_process("mongod-config", config_port, config_argv);
+
+  unsigned int distributed_port = config->get_uint("plugins/robot-memory/setup/replicated/port");
+  std::string distributed_db_path = StringConversions::resolve_path(config->get_string("plugins/robot-memory/setup/replicated/db-path").c_str());
+  prepare_mongo_db_path(distributed_db_path);
+  std::string distributed_port_str = std::to_string(distributed_port);
+  std::string distributed_replset = config->get_string("plugins/robot-memory/setup/replicated/replica-set-name");
+  const char *distributed_argv[] = {"mongod", "--port", distributed_port_str.c_str(),
+      "--dbpath", distributed_db_path.c_str(),
+      "--replSet", distributed_replset.c_str(),NULL};
+  start_mongo_process("mongod-replicated", distributed_port, distributed_argv);
+
+  unsigned int mongos_port = config->get_uint("plugins/robot-memory/setup/mongos/port");
+  std::string mongos_port_str = std::to_string(mongos_port);
+  std::string confighost = "localhost:" + config_port_str;
+  const char *mongos_argv[] = {"mongos", "--port", mongos_port_str.c_str(),
+      "--configdb", confighost.c_str(), NULL};
+  start_mongo_process("mongos", mongos_port, mongos_argv);
+}
+
+/**
+ * Start a single mongo process
+ */
+void RobotMemorySetup::start_mongo_process(std::string proc_name, unsigned int port, const char *argv[])
+{
+  if (!is_mongo_running(port))
+    {
+      std::string cmd = command_args_tostring(argv);
+      logger->log_info("RobotMemorySetup", "Starting %s process: '%s'", proc_name.c_str(), cmd.c_str());
+      config_mongod = new SubProcess(proc_name.c_str(), argv[0], argv, NULL, logger);
+      logger->log_info("RobotMemorySetup", "Started %s", proc_name.c_str());
+      wait_until_started(port, cmd);
+    }
 }
 
 /**
diff --git a/src/plugins/robot-memory/robot_memory_setup.h b/src/plugins/robot-memory/robot_memory_setup.h
index c3bf1ea..d4d0196 100644
--- a/src/plugins/robot-memory/robot_memory_setup.h
+++ b/src/plugins/robot-memory/robot_memory_setup.h
@@ -47,6 +47,7 @@ class RobotMemorySetup
     bool is_mongo_running(unsigned int port);
     void wait_until_started(unsigned int port, std::string cmd, int timout = 15000000);
     void prepare_mongo_db_path(std::string path);
+    void start_mongo_process(std::string proc_name, unsigned int port, const char *argv[]);
 
     fawkes::SubProcess *local_mongod;
     fawkes::SubProcess *config_mongod;

- *commit* a03d31e17ffb0d8a1d5ddb1a791c5d5f4dddab8b - - - - - - - - - -
Author:  Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
Date:    Fri Sep 23 19:16:30 2016 +0200
Subject: clips-robot-memory: add remove function

 .../clips_robot_memory_thread.cpp                  |   15 ++++++++++++++-
 .../clips-robot-memory/clips_robot_memory_thread.h |    1 +
 2 files changed, 15 insertions(+), 1 deletions(-)

_Diff for modified files_:
diff --git a/src/plugins/clips-robot-memory/clips_robot_memory_thread.cpp b/src/plugins/clips-robot-memory/clips_robot_memory_thread.cpp
index 7b5c589..eebe069 100644
--- a/src/plugins/clips-robot-memory/clips_robot_memory_thread.cpp
+++ b/src/plugins/clips-robot-memory/clips_robot_memory_thread.cpp
@@ -78,6 +78,7 @@ ClipsRobotMemoryThread::clips_context_init(const std::string &env_name,
   clips->add_function("robmem-update", sigc::slot<void, std::string, void *, CLIPS::Value>(sigc::mem_fun(*this, &ClipsRobotMemoryThread::clips_robotmemory_update)));
   clips->add_function("robmem-replace", sigc::slot<void, std::string, void *, CLIPS::Value>(sigc::mem_fun(*this, &ClipsRobotMemoryThread::clips_robotmemory_replace)));
   clips->add_function("robmem-query", sigc::slot<CLIPS::Value, std::string, void *>(sigc::mem_fun(*this, &ClipsRobotMemoryThread::clips_robotmemory_query)));
+  clips->add_function("robmem-remove", sigc::slot<void, std::string, void *>(sigc::mem_fun(*this, &ClipsRobotMemoryThread::clips_robotmemory_remove)));
   clips->add_function("robmem-query-sort", sigc::slot<CLIPS::Value, std::string, void *, void *>(sigc::mem_fun(*this, &ClipsRobotMemoryThread::clips_robotmemory_query_sort)));
   clips->add_function("robmem-cursor-destroy", sigc::slot<void, void *>(sigc::mem_fun(*this, &ClipsRobotMemoryThread::clips_robotmemory_cursor_destroy)));
   clips->add_function("robmem-cursor-more", sigc::slot<CLIPS::Value, void *>(sigc::mem_fun(*this, &ClipsRobotMemoryThread::clips_robotmemory_cursor_more)));
@@ -411,7 +412,7 @@ ClipsRobotMemoryThread::clips_robotmemory_query_sort(std::string collection, voi
     return CLIPS::Value(new std::unique_ptr<mongo::DBClientCursor>(std::move(c)),
                         CLIPS::TYPE_EXTERNAL_ADDRESS);
 #else
-    std::auto_ptr<mongo::DBClientCursor> c = mongodb_->query(collection, q);
+    std::auto_ptr<mongo::DBClientCursor> c = robot_memory->query(collection, q);
 
     return CLIPS::Value(new std::auto_ptr<mongo::DBClientCursor>(c),
                         CLIPS::TYPE_EXTERNAL_ADDRESS);
@@ -423,6 +424,18 @@ ClipsRobotMemoryThread::clips_robotmemory_query_sort(std::string collection, voi
   }
 }
 
+void
+ClipsRobotMemoryThread::clips_robotmemory_remove(std::string collection, void *bson)
+{
+  mongo::BSONObjBuilder *b = static_cast<mongo::BSONObjBuilder *>(bson);
+  try {
+    mongo::Query q(b->asTempObj());
+    robot_memory->remove(q, collection);
+  } catch (mongo::DBException &e) {
+    logger->log_warn("MongoDB", "Remove failed: %s", e.what());
+  }
+}
+
 CLIPS::Value
 ClipsRobotMemoryThread::clips_robotmemory_query(std::string collection, void *bson)
 {
diff --git a/src/plugins/clips-robot-memory/clips_robot_memory_thread.h b/src/plugins/clips-robot-memory/clips_robot_memory_thread.h
index 6b43d48..31f9c87 100644
--- a/src/plugins/clips-robot-memory/clips_robot_memory_thread.h
+++ b/src/plugins/clips-robot-memory/clips_robot_memory_thread.h
@@ -83,6 +83,7 @@ class ClipsRobotMemoryThread
                                CLIPS::Value &query, bool upsert);
   CLIPS::Value  clips_robotmemory_query_sort(std::string collection, void *bson, void *bson_sort);
   CLIPS::Value  clips_robotmemory_query(std::string collection, void *bson);
+  void          clips_robotmemory_remove(std::string collection, void *bson);
   CLIPS::Value  clips_robotmemory_cursor_more(void *cursor);
   CLIPS::Value  clips_robotmemory_cursor_next(void *cursor);
   void          clips_robotmemory_cursor_destroy(void *cursor);

- *commit* 291d8475b466d01f1be661bcc8a5b1a96f923ef4 - - - - - - - - - -
Author:  Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
Date:    Wed Sep 28 13:06:57 2016 +0200
Subject: robot-memory: config sharded cluster automatically

 src/plugins/robot-memory/robot_memory_setup.cpp |   31 +++++++++++++++++++++++
 src/plugins/robot-memory/robot_memory_setup.h   |    1 +
 2 files changed, 32 insertions(+), 0 deletions(-)

_Diff for modified files_:
diff --git a/src/plugins/robot-memory/robot_memory_setup.cpp b/src/plugins/robot-memory/robot_memory_setup.cpp
index 1b78571..6b45bf4 100644
--- a/src/plugins/robot-memory/robot_memory_setup.cpp
+++ b/src/plugins/robot-memory/robot_memory_setup.cpp
@@ -54,6 +54,7 @@ void RobotMemorySetup::setup_mongods()
   if(!config->get_bool("plugins/robot-memory/setup/distributed"))
     return;
 
+  //start config server
   unsigned int config_port = config->get_uint("plugins/robot-memory/setup/config/port");
   std::string db_path = StringConversions::resolve_path(config->get_string("plugins/robot-memory/setup/config/db-path").c_str());
   prepare_mongo_db_path(db_path);
@@ -62,6 +63,7 @@ void RobotMemorySetup::setup_mongods()
       config_port_str.c_str(), "--dbpath", db_path.c_str(), NULL};
   start_mongo_process("mongod-config", config_port, config_argv);
 
+  //start own part of replica set
   unsigned int distributed_port = config->get_uint("plugins/robot-memory/setup/replicated/port");
   std::string distributed_db_path = StringConversions::resolve_path(config->get_string("plugins/robot-memory/setup/replicated/db-path").c_str());
   prepare_mongo_db_path(distributed_db_path);
@@ -72,12 +74,23 @@ void RobotMemorySetup::setup_mongods()
       "--replSet", distributed_replset.c_str(),NULL};
   start_mongo_process("mongod-replicated", distributed_port, distributed_argv);
 
+  //configure replica set
+  std::string repl_config = "{_id:'" + distributed_replset + "', members:"
+      + config->get_string("plugins/robot-memory/setup/replicated/replica-set-members") + "}";
+  run_mongo_command(distributed_port, std::string("{replSetInitiate:" + repl_config + "}"));
+
+  //start mongos for accessing
   unsigned int mongos_port = config->get_uint("plugins/robot-memory/setup/mongos/port");
   std::string mongos_port_str = std::to_string(mongos_port);
   std::string confighost = "localhost:" + config_port_str;
   const char *mongos_argv[] = {"mongos", "--port", mongos_port_str.c_str(),
       "--configdb", confighost.c_str(), NULL};
   start_mongo_process("mongos", mongos_port, mongos_argv);
+
+  //configure mongos (add parts of the sharded cluster)
+  run_mongo_command(mongos_port, std::string("{addShard: 'localhost:" + local_port_str + "'}"), "host already used");
+  run_mongo_command(mongos_port, std::string("{addShard: '" + distributed_replset +
+    "/localhost:" + distributed_port_str + "'}"), "host already used");
 }
 
 /**
@@ -171,3 +184,21 @@ void RobotMemorySetup::prepare_mongo_db_path(std::string path)
   std::string command = "mkdir -p " + path;
   popen(command.c_str(), "r");
 }
+
+void RobotMemorySetup::run_mongo_command(unsigned int port, std::string command,
+  std::string err_msg_to_ignore)
+{
+  std::string errmsg;
+  mongo::DBClientConnection con(false);
+  bool could_connect = con.connect(std::string("localhost:" + std::to_string(port)), errmsg);
+  if(!could_connect)
+  {
+    std::string err_msg = "Could not connect to mongo process to execute command: "+ errmsg;
+    throw PluginLoadException("robot-memory", err_msg.c_str());
+  }
+  mongo::BSONObj res;
+  logger->log_info("RobotMemorySetup", "Executing db command: %s", command.c_str());
+  con.runCommand("admin", mongo::fromjson(command), res);
+  if(res.getField("ok").Double() == 0.0 && res.getField("errmsg").String().compare(err_msg_to_ignore) != 0)
+    throw PluginLoadException("robot-memory", std::string("Running DB command " + command + " failed: " + res.toString()).c_str());
+}
diff --git a/src/plugins/robot-memory/robot_memory_setup.h b/src/plugins/robot-memory/robot_memory_setup.h
index d4d0196..06987ed 100644
--- a/src/plugins/robot-memory/robot_memory_setup.h
+++ b/src/plugins/robot-memory/robot_memory_setup.h
@@ -48,6 +48,7 @@ class RobotMemorySetup
     void wait_until_started(unsigned int port, std::string cmd, int timout = 15000000);
     void prepare_mongo_db_path(std::string path);
     void start_mongo_process(std::string proc_name, unsigned int port, const char *argv[]);
+    void run_mongo_command(unsigned int port, std::string command, std::string err_msg_to_ignore="");
 
     fawkes::SubProcess *local_mongod;
     fawkes::SubProcess *config_mongod;

- *commit* 0ba989a955956994feb728db46859943183c089f - - - - - - - - - -
Author:  Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
Date:    Wed Sep 28 14:51:15 2016 +0200
Subject: robot-memory: set read preference 'nearest' to read locally

 src/plugins/robot-memory/robot_memory.cpp       |    3 +++
 src/plugins/robot-memory/robot_memory_setup.cpp |    8 +++++---
 2 files changed, 8 insertions(+), 3 deletions(-)

_Diff for modified files_:
diff --git a/src/plugins/robot-memory/robot_memory.cpp b/src/plugins/robot-memory/robot_memory.cpp
index dce9510..23c4b43 100644
--- a/src/plugins/robot-memory/robot_memory.cpp
+++ b/src/plugins/robot-memory/robot_memory.cpp
@@ -105,6 +105,9 @@ QResCursor RobotMemory::query(Query query, std::string collection)
 //    gen_blackboard_data(query.getFilter()["bbinterface"].String());
 //  }
 
+  //set read preference of query to nearest to read from the local replica set member first
+  query.readPref(ReadPreference_Nearest, BSONArray());
+
   //actually execute query
   QResCursor cursor;
   try{
diff --git a/src/plugins/robot-memory/robot_memory_setup.cpp b/src/plugins/robot-memory/robot_memory_setup.cpp
index 6b45bf4..0968af9 100644
--- a/src/plugins/robot-memory/robot_memory_setup.cpp
+++ b/src/plugins/robot-memory/robot_memory_setup.cpp
@@ -77,7 +77,9 @@ void RobotMemorySetup::setup_mongods()
   //configure replica set
   std::string repl_config = "{_id:'" + distributed_replset + "', members:"
       + config->get_string("plugins/robot-memory/setup/replicated/replica-set-members") + "}";
-  run_mongo_command(distributed_port, std::string("{replSetInitiate:" + repl_config + "}"));
+  run_mongo_command(distributed_port, std::string("{replSetInitiate:" + repl_config + "}"), "already initialized");
+  //wait for replica set initialization and election
+  usleep(1000000);
 
   //start mongos for accessing
   unsigned int mongos_port = config->get_uint("plugins/robot-memory/setup/mongos/port");
@@ -199,6 +201,6 @@ void RobotMemorySetup::run_mongo_command(unsigned int port, std::string command,
   mongo::BSONObj res;
   logger->log_info("RobotMemorySetup", "Executing db command: %s", command.c_str());
   con.runCommand("admin", mongo::fromjson(command), res);
-  if(res.getField("ok").Double() == 0.0 && res.getField("errmsg").String().compare(err_msg_to_ignore) != 0)
-    throw PluginLoadException("robot-memory", std::string("Running DB command " + command + " failed: " + res.toString()).c_str());
+//  if(res.getField("ok").Double() == 0.0 && res.getField("errmsg").String().compare(err_msg_to_ignore) != 0)
+//    throw PluginLoadException("robot-memory", std::string("Running DB command " + command + " failed: " + res.toString()).c_str());
 }

- *commit* 365c95966cad4aac69ea73323cec4f2098ab1531 - - - - - - - - - -
Author:  Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
Date:    Wed Sep 28 14:53:39 2016 +0200
Subject: robot-memory-test: use simulated replica set in tests

 src/tools/gtest/gtest_fawkes.cpp |    8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)

_Diff for modified files_:
diff --git a/src/tools/gtest/gtest_fawkes.cpp b/src/tools/gtest/gtest_fawkes.cpp
index 390ec1f..6c89db3 100644
--- a/src/tools/gtest/gtest_fawkes.cpp
+++ b/src/tools/gtest/gtest_fawkes.cpp
@@ -45,15 +45,19 @@ int main(int argc, char **argv) {
 
     //init arguments to start fawkes with
     char **fawkes_argv;
-    fawkes_argv = new char*[3];
+    fawkes_argv = new char*[5];
     fawkes_argv[0] = new char[6];
     strcpy(fawkes_argv[0], "fawkes");
     fawkes_argv[1] = new char[2];
     strcpy(fawkes_argv[1], "-p");
     fawkes_argv[2] = new char[128];
     strcpy(fawkes_argv[2], "m-robot-memory,robot_memory_test");
+    fawkes_argv[3] = new char[2];
+    strcpy(fawkes_argv[3], "-c");
+    fawkes_argv[4] = new char[128];
+    strcpy(fawkes_argv[4], "gazsim-configurations/default/robotino1.yaml");
 
-    if (! fawkes::runtime::init(3, fawkes_argv, retval)) {
+    if (! fawkes::runtime::init(5, fawkes_argv, retval)) {
       return retval;
     }
     fawkes::runtime::run();

- *commit* d5cc3a8454a34cb3661f409dcfd67262fb5f5b0d - - - - - - - - - -
Author:  Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
Date:    Wed Sep 28 15:47:38 2016 +0200
Subject: robot-memory: made setup robust against missing db+shards already added

 src/plugins/robot-memory/robot_memory_setup.cpp |   20 +++++++++++++-------
 src/plugins/robot-memory/robot_memory_setup.h   |    3 ++-
 2 files changed, 15 insertions(+), 8 deletions(-)

_Diff for modified files_:
diff --git a/src/plugins/robot-memory/robot_memory_setup.cpp b/src/plugins/robot-memory/robot_memory_setup.cpp
index 0968af9..ec4d27b 100644
--- a/src/plugins/robot-memory/robot_memory_setup.cpp
+++ b/src/plugins/robot-memory/robot_memory_setup.cpp
@@ -79,7 +79,8 @@ void RobotMemorySetup::setup_mongods()
       + config->get_string("plugins/robot-memory/setup/replicated/replica-set-members") + "}";
   run_mongo_command(distributed_port, std::string("{replSetInitiate:" + repl_config + "}"), "already initialized");
   //wait for replica set initialization and election
-  usleep(1000000);
+  usleep(3000000);
+  run_mongo_command(distributed_port, std::string("{create: '" + distributed_replset + ".config'}"), "collection already exists");
 
   //start mongos for accessing
   unsigned int mongos_port = config->get_uint("plugins/robot-memory/setup/mongos/port");
@@ -90,9 +91,13 @@ void RobotMemorySetup::setup_mongods()
   start_mongo_process("mongos", mongos_port, mongos_argv);
 
   //configure mongos (add parts of the sharded cluster)
-  run_mongo_command(mongos_port, std::string("{addShard: 'localhost:" + local_port_str + "'}"), "host already used");
-  run_mongo_command(mongos_port, std::string("{addShard: '" + distributed_replset +
-    "/localhost:" + distributed_port_str + "'}"), "host already used");
+  mongo::BSONObj current_shards =  run_mongo_command(mongos_port, std::string("{listShards:1}"));
+  if(current_shards.getField("shards").Array().size() == 0)
+  {
+    run_mongo_command(mongos_port, std::string("{addShard: 'localhost:" + local_port_str + "'}"), "host already used");
+    run_mongo_command(mongos_port, std::string("{addShard: '" + distributed_replset +
+      "/localhost:" + distributed_port_str + "'}"), "host already used");
+  }
 }
 
 /**
@@ -187,7 +192,7 @@ void RobotMemorySetup::prepare_mongo_db_path(std::string path)
   popen(command.c_str(), "r");
 }
 
-void RobotMemorySetup::run_mongo_command(unsigned int port, std::string command,
+mongo::BSONObj RobotMemorySetup::run_mongo_command(unsigned int port, std::string command,
   std::string err_msg_to_ignore)
 {
   std::string errmsg;
@@ -201,6 +206,7 @@ void RobotMemorySetup::run_mongo_command(unsigned int port, std::string command,
   mongo::BSONObj res;
   logger->log_info("RobotMemorySetup", "Executing db command: %s", command.c_str());
   con.runCommand("admin", mongo::fromjson(command), res);
-//  if(res.getField("ok").Double() == 0.0 && res.getField("errmsg").String().compare(err_msg_to_ignore) != 0)
-//    throw PluginLoadException("robot-memory", std::string("Running DB command " + command + " failed: " + res.toString()).c_str());
+  if(res.getField("ok").Double() == 0.0 && res.getField("errmsg").String().compare(err_msg_to_ignore) != 0)
+    throw PluginLoadException("robot-memory", std::string("Running DB command " + command + " failed: " + res.toString()).c_str());
+  return res;
 }
diff --git a/src/plugins/robot-memory/robot_memory_setup.h b/src/plugins/robot-memory/robot_memory_setup.h
index 06987ed..1764c4d 100644
--- a/src/plugins/robot-memory/robot_memory_setup.h
+++ b/src/plugins/robot-memory/robot_memory_setup.h
@@ -26,6 +26,7 @@
 #include <logging/logger.h>
 #include <utils/sub_process/proc.h>
 #include <utils/misc/string_commands.h>
+#include <mongo/client/dbclient.h>
 
 /** @class RobotMemorySetup  robot_memory_setup.h
  *
@@ -48,7 +49,7 @@ class RobotMemorySetup
     void wait_until_started(unsigned int port, std::string cmd, int timout = 15000000);
     void prepare_mongo_db_path(std::string path);
     void start_mongo_process(std::string proc_name, unsigned int port, const char *argv[]);
-    void run_mongo_command(unsigned int port, std::string command, std::string err_msg_to_ignore="");
+    mongo::BSONObj run_mongo_command(unsigned int port, std::string command, std::string err_msg_to_ignore="");
 
     fawkes::SubProcess *local_mongod;
     fawkes::SubProcess *config_mongod;

- *commit* 19d0fbfafa5901603ac07a8dd13cc6021f0350a0 - - - - - - - - - -
Author:  Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
Date:    Wed Sep 28 16:18:59 2016 +0200
Subject: robot-memory: use default db name when modified for simulation

 src/plugins/robot-memory/robot_memory.cpp |   33 ++++++++++++++++++----------
 src/plugins/robot-memory/robot_memory.h   |    2 +
 2 files changed, 23 insertions(+), 12 deletions(-)

_Diff for modified files_:
diff --git a/src/plugins/robot-memory/robot_memory.cpp b/src/plugins/robot-memory/robot_memory.cpp
index 23c4b43..5fba663 100644
--- a/src/plugins/robot-memory/robot_memory.cpp
+++ b/src/plugins/robot-memory/robot_memory.cpp
@@ -81,8 +81,7 @@ void RobotMemory::init()
 
 QResCursor RobotMemory::query(Query query, std::string collection)
 {
-  if(collection == "")
-    collection = default_collection_;
+  check_collection_name(collection);
   log_deb(std::string("Executing Query "+ query.toString() +" on collection "+collection));
 
   //only one query at a time
@@ -123,8 +122,7 @@ QResCursor RobotMemory::query(Query query, std::string collection)
 
 int RobotMemory::insert(BSONObj obj, std::string collection)
 {
-  if(collection == "")
-    collection = default_collection_;
+  check_collection_name(collection);
 
   log_deb(std::string("Executing Query "+ obj.toString() + " on collection " + collection));
 
@@ -151,10 +149,7 @@ int RobotMemory::insert(std::string obj_str, std::string collection)
 
 int RobotMemory::update(Query query, BSONObj update, std::string collection, bool upsert)
 {
-  if(collection == "")
-  {
-    collection = default_collection_;
-  }
+  check_collection_name(collection);
   log_deb(std::string("Executing Update "+update.toString()+" for query "+query.toString()+" on collection "+ collection));
 
   //only one query at a time
@@ -178,10 +173,7 @@ int RobotMemory::update(Query query, std::string update_str, std::string collect
 
 int RobotMemory::remove(Query query, std::string collection)
 {
-  if(collection == "")
-  {
-    collection = default_collection_;
-  }
+  check_collection_name(collection);
   log_deb(std::string("Executing Remove "+query.toString()+" on collection "+collection));
 
   //only one query at a time
@@ -399,3 +391,20 @@ RobotMemory::remove_field(Query &q, std::string what)
   //override
   q = Query(b.obj());
 }
+
+/**
+ * Check if collection name is valid and correct it if necessary
+ */
+void
+RobotMemory::check_collection_name(std::string &collection)
+{
+  if(collection == "")
+  {
+      collection = default_collection_;
+  }
+  else if(default_collection_ != "robmem" && collection.find("robmem.") == 1)
+  {
+    //change used database name (e.g. for the case of multiple simulated dababases)
+    collection.replace(0, 6, default_collection_);
+  }
+}
diff --git a/src/plugins/robot-memory/robot_memory.h b/src/plugins/robot-memory/robot_memory.h
index 2ad892b..4de84fd 100644
--- a/src/plugins/robot-memory/robot_memory.h
+++ b/src/plugins/robot-memory/robot_memory.h
@@ -89,6 +89,8 @@ class RobotMemory
     void set_fields(mongo::BSONObj &obj, std::string what);
     void set_fields(mongo::Query &q, std::string what);
     void remove_field(mongo::Query &q, std::string what);
+
+    void check_collection_name(std::string &collection);
 };
 
 #endif /* FAWKES_SRC_PLUGINS_ROBOT_MEMORY_ROBOT_MEMORY_H_ */

- *commit* 5173577ed6812e699e22143881e221a09629e707 - - - - - - - - - -
Author:  Frederik Zwilling <zwilling at kbsg.rwth-aachen.de>
Date:    Wed Sep 28 18:35:30 2016 +0200
Subject: robot-memory: added dependencies in plugins/Makefile

 src/plugins/Makefile |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

_Diff for modified files_:
diff --git a/src/plugins/Makefile b/src/plugins/Makefile
index e400377..61072d8 100644
--- a/src/plugins/Makefile
+++ b/src/plugins/Makefile
@@ -38,10 +38,11 @@ katana jaco: openrave
 amcl colli navgraph perception robotino: ros
 mongodb_log: mongodb
 mongodb: rrd
-clips-navgraph clips-agent clips-protobuf clips-tf: clips
+clips-navgraph clips-agent clips-protobuf clips-tf clips-robot-memory: clips
 clips-navgraph navgraph-clusters: navgraph
 clips-webview: clips webview
 clips-ros: clips ros
+clips-robot-memory: robot-memory
 gazebo: robotino
 skiller: navgraph
 perception: mongodb




-- 
Fawkes Robotics Framework                 http://www.fawkesrobotics.org


More information about the fawkes-commits mailing list