Skip to content

Agent Types

Jump to bottom
Erik Harpstead edited this page Jul 25, 2020 · 6 revisions

The Apprentice Learner Architecture supports a number of different agent implementations currently. Below we describe the general differences between them and what parameters they support for creation.

ModularAgent

The ModularAgent type is the agent type you will use in most cases. It represents the current best implementation of the Apprentice Learner API as a modular architecture that can represent a number of different learners. The ModularAgent's args object contains the following optional fields:

  • when_learner - Specifies which implementation to use for when learning. Must be one of the following:
    • decisiontree - A standard decision tree classifier. Uses the implementation of DecisionTreeClassifier from scikit.
    • cobweb - A hierarchical concept formation tree that can be applied to mixed nominal and numeric data. Uses the implementation of Cobweb3 from the concept_formation library.
    • trestle - A hierarchical concept formation tree that can handle structured input data in addition to mixed nominal and numeric data. Uses the implementation of Trestle from the concept_formation library.
  • where_learner - Specifies which implementation to use for where learning. Must be one of the following:
    • versionspace - Uses and implementation of Mitchell's Version Space Algorithm.
    • fastmostspecific - An optimized version of the Most Specific learner below.
    • mostspecific- Creates retrieval patterns based on the most specific description of where values are.
    • stateresponselearner - A state response learner.
    • relationallearner - A Relational learner.
    • specifictogeneral - Creates retrieval patterns that start from a specific description and then generalizes patterns as more examples come in.
  • heuristic_learner - Specifies which implementation to use for which learning. Must be one of the following:
    • proportioncorrect - Resolves skill conflicts by favoring the skill with the highest percentage of being correct in the past.
    • totalcorrect - Resolves skill conflicts by favoring the skill with the most prior correct applications, without considering incorrect applications.
  • how_cull_rule - Specifies which implementation to use to limit the How Search. Must be one of the following:
    • first - how search returns the first explanation that successfully covers the demonstration.
    • mostparsimonious - how search returns the shortest explanation that successfully covers the demonstration
    • all - how search returns all explanations
  • planner - Specifies which planner implementation to use for How Search. Must be one of the following:
    • fo_planner - A first order logic planner. This is the original implementation of the Apprentice Learner, used in most published papers to date, but can be slower in some situations. This planner makes use of operators specified in the apprentice.working_memory.fo_planner_operators module.
    • vectorized - A vectorized implementation of a planner. This implementation is faster but can currently only be applied to numerical data and a subset of operators.
    • numba / numbert - The numbert planner. A highly performant planner based on the numba library meant to replace to vectorized_planner. This planner makes use of operators specified in the apprentice.working_memory.numba_operators module.
  • function_set - A list of names of operators to be used during explanation search and how learning. Operators must be pre-defined in a location based on the planner being used. See the planner argument above.
  • feature_set - A list of names of operators to be used during relational inference. Operators must be pre-defined in a location based on the planner being used. See the planner argument above.
  • search_depth - Specifies the depth of explanation to use for how search.
  • numerical_epsilon - Specifies a maximum difference between 2 float values for numbers to be considered equal during planner searches.

ExpertaAgent

The ExpertaAgent is a newer agent implementation that makes use of the Experta expert systems framework. The agent does not adhere as closely to the learning process decomposition as the ModularAgent but its expert systems approach allows it greater flexibility to do things like forward planning and handling delayed feedback. An additional constraint of the ExpertaAgent is that it requires both current and next states to be provided for training. If you are using the AL_Train harness this constraint is already account for. The ExpertaAgent's args object contains the following optional fields:

  • action_penalty - The cost associated with taking an action in forwardly planning. Used to penalize overly complex plans. Must be a float,
  • request_epsilon - The epsilon used during requests. Must be a float.
  • train_epsilon - The epsilon used during training. Must be a float.
  • gamma - The gamma value. Must be a float
  • negative_actions - A flag for whether the agent is allowed to take negative actions. Must be true or false.
  • chunking - A flag for whether the agent should apply chunking or not. Must be true or false.
  • prior_skills - An object specifying the existing skills the agent should start with. Specified as "[SKILL_NAME]": true/false. Skills must be pre-defined as Experta rules inside the CoreKnoweldgeEngine in the apprentice.working_memory.experta_skills module.

WhereWhenHowNoFoa

The WhereWhenHowNoFoa is an older implementation of the Apprentice Learner on which the ModularAgent is based. You will likely never use it directly unless you are trying to replicate an old finding or doing some regression testing. The WhereWhenHowNoFoa's args object contains the following optional fields:

  • when_learner - Specifies which implementation to use for when learning. Must be one of the following:
    • decisiontree - A standard decision tree classifier. Uses the implementation of DecisionTreeClassifier from scikit.
    • cobweb - A hierarchical concept formation tree that can be applied to mixed nominal and numeric data. Uses the implementation of Cobweb3 from the concept_formation library.
    • trestle - A hierarhcical concept formation tree that can handle structured input data in addition to mixed nominal and numeric data. Uses the implementation of Trestle from the concept_formation library.
  • where_learner - Specifies which implementation to use for where learning. Must be one of the following:
    • mostspecific- Creates retrieval patterns based on the most specific description of where values are.
    • specifictogeneral - Creates retrieval patterns that start from a specific description and then generalizes patterns as more examples come in.
  • search_depth - Specifies the depth of explanation to use for how search.
  • numerical_epsilon - Specifies a maximum difference between 2 float values for numbers to be considered equal during planner searches.

RLAgent

The RLAgent is an early implementation of a Reinforcement Learning based agent. It is currently an experimental agent so it is not likely to be used. The RLAgent's args object does not support any fields.

Memo

The Memo agent is a basic agent that simply memorizes an association of state-action pairs to rewards on train and responds with the highest reward action for a given state on request. It mainly exists for baseline comparisons and testing.

Stub

Like the Memo agent, the Stub agent implements each of the core API targets as no-ops. It also mainly exists for API testing purposes.

Clone this wiki locally