Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nonlinear Hybrid #1263

Merged
merged 42 commits into from
Aug 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
2927d92
add HybridNonlinearFactor and nonlinear HybridFactorGraph
varunagrawal May 28, 2022
78ea90b
Add MixtureFactor for nonlinear factor types
varunagrawal May 28, 2022
e91a354
convert to cpp
varunagrawal May 29, 2022
9cbd2ef
Base Hybrid Factor Graph
varunagrawal May 29, 2022
01b9a65
make GaussianMixtureFactor a subclass of HybridGaussianFactor
varunagrawal May 29, 2022
fe0d666
HybridFactorGraph fixes
varunagrawal May 29, 2022
cdd030b
Make MixtureFactor only work with NonlinearFactors and make some impr…
varunagrawal May 29, 2022
08fab8a
HybridNonlinearFactor linearize method
varunagrawal May 29, 2022
9279bd6
push_back for GaussianHybridFactor
varunagrawal May 29, 2022
3274cb1
clean up testHybridFactorGraph, need to add more tests
varunagrawal May 29, 2022
6c36b2c
GaussianHybridFactorGraph inherits from HybridFactorGraph
varunagrawal May 29, 2022
85f4b48
Improvements to GaussianHybridFactorGraph, make MixtureFactor a subcl…
varunagrawal May 30, 2022
53e8c32
Add NonlinearHybridFactorGraph class
varunagrawal May 30, 2022
7e18277
fix base class
varunagrawal May 30, 2022
119679a
linearize returns object instead of pointer
varunagrawal May 30, 2022
3212dde
remove unneeded method
varunagrawal May 30, 2022
0c16799
GaussianMixtureFactor inherits from HybridFactor
varunagrawal May 30, 2022
e711a62
More tests working
varunagrawal May 30, 2022
3780b8c
Merge branch 'develop' into feature/nonlinear-hybrid
varunagrawal Jul 26, 2022
8ddc2ea
rename to HybridNonlinearFactorGraph
varunagrawal Jul 26, 2022
7a55341
add IsGaussian template check
varunagrawal Jul 26, 2022
43c28e7
renaming fixes
varunagrawal Jul 26, 2022
987448f
remove derived push_back in HybridNonlinearFactorGraph and HybridFact…
varunagrawal Jul 26, 2022
8471c97
add nonlinear switching system tests
varunagrawal Jul 30, 2022
8907922
get more nonlinear tests to work and make some updates
varunagrawal Aug 1, 2022
16124f3
get all but 2 tests passing
varunagrawal Aug 2, 2022
ee124c3
fix discrete only elimination (use EliminateForMPE)
varunagrawal Aug 2, 2022
b39c231
all tests pass!!!
varunagrawal Aug 2, 2022
0f732d7
fix discrete conditional test
varunagrawal Aug 3, 2022
6670779
Wrap DiscreteLookupTable
varunagrawal Aug 3, 2022
92a5868
Merge branch 'develop' into feature/nonlinear-hybrid
varunagrawal Aug 3, 2022
db56909
Merge branch 'hybrid/pruning' into feature/nonlinear-hybrid
varunagrawal Aug 3, 2022
5965d8f
change discrete key variable from C to M
varunagrawal Aug 8, 2022
f5e046f
split HybridNonlinearFactorGraph to .h and .cpp
varunagrawal Aug 8, 2022
51d2f07
fix printing and key bug in MixtureFactor linearize
varunagrawal Aug 8, 2022
a3eacaa
fix adding priors in Switching
varunagrawal Aug 8, 2022
588f56e
HybridGaussianISAM unit tests
varunagrawal Aug 8, 2022
60c88e3
fix print tests
varunagrawal Aug 10, 2022
4ea897c
cleaner printing
varunagrawal Aug 10, 2022
fbceda3
got some more tests working
varunagrawal Aug 10, 2022
0e4db30
use templetized constructor for MixtureFactor
varunagrawal Aug 10, 2022
2a974a4
Address review comments
varunagrawal Aug 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions gtsam/discrete/discrete.i
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,16 @@ class DiscreteBayesTree {
};

#include <gtsam/discrete/DiscreteLookupDAG.h>

class DiscreteLookupTable : gtsam::DiscreteConditional{
DiscreteLookupTable(size_t nFrontals, const gtsam::DiscreteKeys& keys,
const gtsam::DecisionTreeFactor::ADT& potentials);
void print(
const std::string& s = "Discrete Lookup Table: ",
const gtsam::KeyFormatter& keyFormatter = gtsam::DefaultKeyFormatter) const;
size_t argmax(const gtsam::DiscreteValues& parentsValues) const;
};

class DiscreteLookupDAG {
DiscreteLookupDAG();
void push_back(const gtsam::DiscreteLookupTable* table);
Expand Down
2 changes: 1 addition & 1 deletion gtsam/hybrid/GaussianMixtureFactor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ GaussianMixtureFactor GaussianMixtureFactor::FromFactors(
void GaussianMixtureFactor::print(const std::string &s,
const KeyFormatter &formatter) const {
HybridFactor::print(s, formatter);
std::cout << "]{\n";
std::cout << "{\n";
factors_.print(
"", [&](Key k) { return formatter(k); },
[&](const GaussianFactor::shared_ptr &gf) -> std::string {
Expand Down
2 changes: 1 addition & 1 deletion gtsam/hybrid/GaussianMixtureFactor.h
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@

#include <gtsam/discrete/DecisionTree.h>
#include <gtsam/discrete/DiscreteKey.h>
#include <gtsam/hybrid/HybridFactor.h>
#include <gtsam/hybrid/HybridGaussianFactor.h>
#include <gtsam/linear/GaussianFactor.h>

namespace gtsam {
Expand Down
5 changes: 4 additions & 1 deletion gtsam/hybrid/HybridBayesTree.h
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,10 @@ class GTSAM_EXPORT HybridBayesTree : public BayesTree<HybridBayesTreeClique> {
/**
* @brief Class for Hybrid Bayes tree orphan subtrees.
*
* This does special stuff for the hybrid case
* This object stores parent keys in our base type factor so that
* eliminating those parent keys will pull this subtree into the
* elimination.
* This does special stuff for the hybrid case.
*
* @tparam CLIQUE
*/
Expand Down
14 changes: 10 additions & 4 deletions gtsam/hybrid/HybridFactor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -89,17 +89,23 @@ void HybridFactor::print(const std::string &s,
if (isContinuous_) std::cout << "Continuous ";
if (isDiscrete_) std::cout << "Discrete ";
if (isHybrid_) std::cout << "Hybrid ";
for (size_t c=0; c<continuousKeys_.size(); c++) {
std::cout << "[";
for (size_t c = 0; c < continuousKeys_.size(); c++) {
std::cout << formatter(continuousKeys_.at(c));
if (c < continuousKeys_.size() - 1) {
std::cout << " ";
} else {
std::cout << "; ";
std::cout << (discreteKeys_.size() > 0 ? "; " : "");
}
}
for(auto && discreteKey: discreteKeys_) {
std::cout << formatter(discreteKey.first) << " ";
for (size_t d = 0; d < discreteKeys_.size(); d++) {
std::cout << formatter(discreteKeys_.at(d).first);
if (d < discreteKeys_.size() - 1) {
std::cout << " ";
}

}
std::cout << "]";
}

} // namespace gtsam
4 changes: 3 additions & 1 deletion gtsam/hybrid/HybridGaussianFactor.h
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ class GTSAM_EXPORT HybridGaussianFactor : public HybridFactor {
using This = HybridGaussianFactor;
using shared_ptr = boost::shared_ptr<This>;

HybridGaussianFactor() = default;

// Explicit conversion from a shared ptr of GF
explicit HybridGaussianFactor(GaussianFactor::shared_ptr other);

Expand All @@ -52,7 +54,7 @@ class GTSAM_EXPORT HybridGaussianFactor : public HybridFactor {

/// GTSAM print utility.
void print(
const std::string &s = "HybridFactor\n",
const std::string &s = "HybridGaussianFactor\n",
const KeyFormatter &formatter = DefaultKeyFormatter) const override;

/// @}
Expand Down
12 changes: 9 additions & 3 deletions gtsam/hybrid/HybridGaussianFactorGraph.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,12 @@ GaussianMixtureFactor::Sum sumFrontals(
} else if (f->isContinuous()) {
deferredFactors.push_back(
boost::dynamic_pointer_cast<HybridGaussianFactor>(f)->inner());

} else if (f->isDiscrete()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change necessary? I remember at this step there should be no pure discrete factors.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah there was one test that was failing without this. You can probably comment it out and find out which one.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember now. If we have discrete only factors it goes to the next condition and errors out since it is not an orphan. That shouldn't happen right? If we only have discrete factors left, we should just be returning a discrete decision tree and not erroring out.

@ProfFan can you please explain what was your rationale here?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think because this is a GaussianMixtureFactor, there should not be any discrete factor added to it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a Hybrid Factor Graph method so a discrete factor can be present.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not opposing to adding this branching. I am a bit worried that people could start to use this function without thinking about the preconditions when this operation is valid. So I prefer to keep this function as-is, or add some comments in the function Doxygen as a warning.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay now this makes sense. In that case, you could have mentioned this before rather than simply saying this is wrong. I am not aware of your mental model and there aren't any comments about the same.

I believe we should still test EliminateHybrid individually. Letting users know what the assumptions are is the right way here, but we should also ensure we handle these kind of edge cases so it makes using the API easier. Can you please comment what would you like me to add as a note in the docstring and I'll add that to this PR?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need to talk about the actual situation that will happen when this is called on an arbitrary factor graph. What I can think of is

Function to eliminate variables **under the following assumptions**:
1. When the ordering is fully continuous, and the graph only contains continuous and hybrid factors
2. When the ordering is fully discrete, and the graph only contains discrete factors
Any usage outside of this is incorrect.

\warning This function is not meant to be used with arbitrary hybrid factor graphs. For example, if there exists continuous parents, and one tries to eliminate a discrete variable (as specified in the ordering), the result will be INCORRECT and there will be NO error raised.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does that sound to you?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me.

// Don't do anything for discrete-only factors
// since we want to eliminate continuous values only.
continue;

} else {
// We need to handle the case where the object is actually an
// BayesTreeOrphanWrapper!
Expand All @@ -106,8 +112,8 @@ GaussianMixtureFactor::Sum sumFrontals(
if (!orphan) {
auto &fr = *f;
throw std::invalid_argument(
std::string("factor is discrete in continuous elimination") +
typeid(fr).name());
std::string("factor is discrete in continuous elimination ") +
demangle(typeid(fr).name()));
}
}
}
Expand Down Expand Up @@ -158,7 +164,7 @@ discreteElimination(const HybridGaussianFactorGraph &factors,
}
}

auto result = EliminateDiscrete(dfg, frontalKeys);
auto result = EliminateForMPE(dfg, frontalKeys);

return {boost::make_shared<HybridConditional>(result.first),
boost::make_shared<HybridDiscreteFactor>(result.second)};
Expand Down
1 change: 1 addition & 0 deletions gtsam/hybrid/HybridGaussianFactorGraph.h
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
#include <gtsam/inference/EliminateableFactorGraph.h>
#include <gtsam/inference/FactorGraph.h>
#include <gtsam/inference/Ordering.h>
#include <gtsam/linear/GaussianFactor.h>

namespace gtsam {

Expand Down
41 changes: 41 additions & 0 deletions gtsam/hybrid/HybridNonlinearFactor.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/* ----------------------------------------------------------------------------

* GTSAM Copyright 2010, Georgia Tech Research Corporation,
* Atlanta, Georgia 30332-0415
* All Rights Reserved
* Authors: Frank Dellaert, et al. (see THANKS for the full author list)

* See LICENSE for the license information

* -------------------------------------------------------------------------- */

/**
* @file HybridNonlinearFactor.cpp
* @date May 28, 2022
* @author Varun Agrawal
*/

#include <gtsam/hybrid/HybridNonlinearFactor.h>

#include <boost/make_shared.hpp>

namespace gtsam {

/* ************************************************************************* */
HybridNonlinearFactor::HybridNonlinearFactor(
const NonlinearFactor::shared_ptr &other)
: Base(other->keys()), inner_(other) {}

/* ************************************************************************* */
bool HybridNonlinearFactor::equals(const HybridFactor &lf, double tol) const {
return Base::equals(lf, tol);
}

/* ************************************************************************* */
void HybridNonlinearFactor::print(const std::string &s,
const KeyFormatter &formatter) const {
HybridFactor::print(s, formatter);
inner_->print("\n", formatter);
};

} // namespace gtsam
62 changes: 62 additions & 0 deletions gtsam/hybrid/HybridNonlinearFactor.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
/* ----------------------------------------------------------------------------

* GTSAM Copyright 2010, Georgia Tech Research Corporation,
* Atlanta, Georgia 30332-0415
* All Rights Reserved
* Authors: Frank Dellaert, et al. (see THANKS for the full author list)

* See LICENSE for the license information

* -------------------------------------------------------------------------- */

/**
* @file HybridNonlinearFactor.h
* @date May 28, 2022
* @author Varun Agrawal
*/

#pragma once

#include <gtsam/hybrid/HybridFactor.h>
#include <gtsam/hybrid/HybridGaussianFactor.h>
#include <gtsam/nonlinear/NonlinearFactor.h>

namespace gtsam {

/**
* A HybridNonlinearFactor is a layer over NonlinearFactor so that we do not
* have a diamond inheritance.
*/
class HybridNonlinearFactor : public HybridFactor {
private:
NonlinearFactor::shared_ptr inner_;

public:
using Base = HybridFactor;
using This = HybridNonlinearFactor;
using shared_ptr = boost::shared_ptr<This>;

// Explicit conversion from a shared ptr of NonlinearFactor
explicit HybridNonlinearFactor(const NonlinearFactor::shared_ptr &other);

/// @name Testable
/// @{

/// Check equality.
virtual bool equals(const HybridFactor &lf, double tol) const override;

/// GTSAM print utility.
void print(
const std::string &s = "HybridNonlinearFactor\n",
const KeyFormatter &formatter = DefaultKeyFormatter) const override;

/// @}

NonlinearFactor::shared_ptr inner() const { return inner_; }

/// Linearize to a HybridGaussianFactor at the linearization point `c`.
boost::shared_ptr<HybridGaussianFactor> linearize(const Values &c) const {
return boost::make_shared<HybridGaussianFactor>(inner_->linearize(c));
}
};
} // namespace gtsam
94 changes: 94 additions & 0 deletions gtsam/hybrid/HybridNonlinearFactorGraph.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
/* ----------------------------------------------------------------------------

* GTSAM Copyright 2010, Georgia Tech Research Corporation,
* Atlanta, Georgia 30332-0415
* All Rights Reserved
* Authors: Frank Dellaert, et al. (see THANKS for the full author list)

* See LICENSE for the license information

* -------------------------------------------------------------------------- */

/**
* @file HybridNonlinearFactorGraph.cpp
* @brief Nonlinear hybrid factor graph that uses type erasure
* @author Varun Agrawal
* @date May 28, 2022
*/

#include <gtsam/hybrid/HybridNonlinearFactorGraph.h>

namespace gtsam {

/* ************************************************************************* */
void HybridNonlinearFactorGraph::add(
boost::shared_ptr<NonlinearFactor> factor) {
FactorGraph::add(boost::make_shared<HybridNonlinearFactor>(factor));
}

/* ************************************************************************* */
void HybridNonlinearFactorGraph::print(const std::string& s,
const KeyFormatter& keyFormatter) const {
// Base::print(str, keyFormatter);
std::cout << (s.empty() ? "" : s + " ") << std::endl;
std::cout << "size: " << size() << std::endl;
for (size_t i = 0; i < factors_.size(); i++) {
std::stringstream ss;
ss << "factor " << i << ": ";
if (factors_[i]) {
factors_[i]->print(ss.str(), keyFormatter);
std::cout << std::endl;
}
}
}

/* ************************************************************************* */
HybridGaussianFactorGraph HybridNonlinearFactorGraph::linearize(
const Values& continuousValues) const {
// create an empty linear FG
HybridGaussianFactorGraph linearFG;

linearFG.reserve(size());

// linearize all hybrid factors
for (auto&& factor : factors_) {
// First check if it is a valid factor
if (factor) {
// Check if the factor is a hybrid factor.
// It can be either a nonlinear MixtureFactor or a linear
// GaussianMixtureFactor.
if (factor->isHybrid()) {
// Check if it is a nonlinear mixture factor
if (auto nlmf = boost::dynamic_pointer_cast<MixtureFactor>(factor)) {
linearFG.push_back(nlmf->linearize(continuousValues));
} else {
linearFG.push_back(factor);
}

// Now check if the factor is a continuous only factor.
} else if (factor->isContinuous()) {
// In this case, we check if factor->inner() is nonlinear since
// HybridFactors wrap over continuous factors.
auto nlhf = boost::dynamic_pointer_cast<HybridNonlinearFactor>(factor);
if (auto nlf =
boost::dynamic_pointer_cast<NonlinearFactor>(nlhf->inner())) {
auto hgf = boost::make_shared<HybridGaussianFactor>(
nlf->linearize(continuousValues));
linearFG.push_back(hgf);
} else {
linearFG.push_back(factor);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ignore if not right) So we do allow pushing linear factors to the Hybrid NLFG? Because I remember that there is a LinearContainerFactor, not sure if we want to do the same here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. This is something we can look at again once we update the type hierarchy for hybrid.

}
// Finally if nothing else, we are discrete-only which doesn't need
// lineariztion.
} else {
linearFG.push_back(factor);
}

} else {
linearFG.push_back(GaussianFactor::shared_ptr());
}
}
return linearFG;
}

} // namespace gtsam
Loading