Title: Hacking Guide to Lyntin Author: Will Guaraldi VERSION ======= $Id: HACKING,v 1.5 2003/11/22 21:45:42 willhelm Exp $ For more up-to-date information, please check the web-site: http://lyntin.sourceforge.net/ NOTES ===== This document provides a rough overview of how things in Lyntin are architected and how they work. The API documentation which is viewable online on the Lyntin site and is in the source code itself which comes with every copy of Lyntin (because it _is_ Lyntin) is where you will find the answers to all your questions and more! If you have questions that you can't find answers to, ask on the lyntin-devl mailing list. Details are on the website. In depth knowledge of the Python programming language is required to really get into Lyntin since it's all written in Python. If you don't know Python, go visit http://www.python.org and familiarize yourself with the language, its constructs. and the Python community. SYNOPSIS ======== Lyntin was built for the hacker mudder. This is a special breed of person who muds and has a series of complex problems they want to solve that aliases and triggers aren't going solve. Tintin has a series of commands that implements a macro language. Building "macros" in this way is complex and solves a small set of problems, but the more complex problems really need a full language and a library to solve. The Tintin way of solving this is to keep adding commands that fill out the macro language to handle string manipulations and such. Lyntin takes a different approach leaving those sorts of things to Python. Lyntin is inextricably intertwined with Python so you can extend Lyntin to do anything without going through hoops: check your email, record URLs you see and open up a browser to visit them, export mudmail, building complex bots, web-servers, AIM clients, automappers.... There are a few different methods of hacking Lyntin to suit your needs. They are briefly outlined here and described in more detail in their respective sections: 1. editing the source code 2. building modules 3. command line Before we continue, we should ramble on about Lyntin's architecture first and how to do things. Before that, we'll cover Lyntin's philosophy. PHILOSOPHY ========== If you enter into the Python interactive mode (by typing "python" at the prompt) and type "import this" you get a list of Python's design philosophies. When working on Lyntin, we have similar thoughts: * Beautiful is better than ugly. * Simple is better than complex. * Readability before most things. * Even readable things need documentation. * Optimization if we have to. * Leave power and choice in the hands of the user. * Get it right the third time--the first two times explore the issue and perhaps implement solutions until the true solution is obvious. * Discussion before implementation. * Possibly specification and test cases before implementation. LYNTIN ARCHITECTURE =================== FILE STRUCTURE -------------- The Lyntin directory structure in CVS looks like this: lyntin40/ |-- lyntin/ | |-- ui/ -- user interfaces | |-- modules/ -- Lyntin modules | |-- scripts/ -- holds the Lyntin startup scripts |-- tools/ -- random stuff for testing and other stuff When you startup Lyntin, you should specify a datadir. I store all my Lyntin data here (command files, session data, AIM history dumps...). DESIGN ------ Lyntin is multi-threaded and as such is built around an event queue. The event queue is synchronized and allows us to have multiple threads going without creating mutexes for everything involved. As such, spin things off into events whenever you can--especially when you're interacting with the various data structures. The engine holds the event queue and drives Lyntin. Most Lyntin things occur via events. The event queue is a synchronized queue and thus gives us a centralized way of synchronizing all the threads accessing Lyntin internals without causing them to block. The engine holds: * the CommandManager * the ConfigManager * the HelpManager * the HistoryManager * all running Sessions * the user interface * the hook data structures * a series of other managers which manage aliases, actions, gags, highlights, substitutes, variables and other such things Managers can manage things on an engine scoping. For example, the HelpManager just has help for all of Lyntin--there's no help topics for a specific session. Managers can also manage things on a session scoping. For example, the ActionManager (in the action plugin) manages data for each session. API --- The API for accessing Lyntin's internals is in the "exported" module. This module contains interfaces to Lyntin's internals with the additional social contract that we Lyntin developers will not change the API between minor versions of Lyntin. Lyntin's internals actually use the exported module for accessing its own internals. The reasons for this being that it gives us a centralized place for fixing behavior bugs. If you plan on building Lyntin modules, use the exported API--it will save you the frustration of dealing with internal changes between Lyntin releases. MANAGERS -------- Managers manage things. Most managers subclass the "manager.Manager" class. It doesn't really provide a lot of functionality, but it allows us to group them all and treat them all the same. Adding new managers is much easier because of this. Also, managers register themselves with the engine via the "exported.add_manager" function. The engine will cycle through registered managers for things like status and persistence. In addition, registered managers get told when the user has created a new session and when the user has ended a session through the "addSession" and "removeSession" methods. For more information, check out the "manager" module in the API. COMMANDS -------- Lyntin comes with a series of commands for manipulating aliases, actions, highlights, and various other things. Both the commands and the managers that hold the data are all defined in various Lyntin modules in the "modules/" subdirectory and are loaded up at Lyntin startup. Lyntin has an extremely powerful argument parser that allows Lyntin module writers to worry about the functionality of their command without having to deal with parsing out the arguments, type-checking, raising exceptions with command syntax help, and data conversion. Commands are stored internally in the "CommandManager". They are global to Lyntin--there are no commands that only exist in the session they were declared in. Commands, however, are executed in a specific session. When they are executed, they are given three arguments: the session object for the session they were executed in, an argument dictionary from the argument parser, and the exact input the user typed. Look at the modules/lyntincmds.py and modules/tintincmds.py modules for command examples. Additionally, check out the Lyntin module repository on http://lyntin.sourceforge.net/ for more examples. ARGUMENT PARSER --------------- tbd. (Anyone who wants to write this, please do.) LYNTIN HELP ----------- Lyntin has a comprehensive help system that can be accessed in-game through the "#help" command. The help material is all organized and manipulated by the "HelpManager". Help is organized as topics which exist in a hierarchy of categories and it comes from a couple of sources: 1) when commands are registered via the exported.add_command function 2) by any module using the exported.add_help function There has been some effort to move the README into in-game help topics and then to actually produce the README file from the in-game help topics. At some point this will allow us to create a Lyntin manual which details all the commands and their syntaxes. USER DATA HANDLING ------------------ User data (sometimes called "input") is received from the "ui.base.BaseUI" subclasses and tossed into an "event.InputEvent". When the engine handles the event, it is first passed to the engine's "handleUserData" method which deals with global things such as splitting the input into a series of separate commands. Read the docstrings for more information. The user input is then passed to the session in question's "handleUserData" method which does some stuff and passes it through the "user_filter_hook". This is where the magic happens. All user input handlers in Lyntin (the alias manager, the variable manager ...) register with the user_filter_hook at various priorities. The "user_filter_hook" then cycles through the user filters in order of priority. For more specific information, read through the docstrings of the methods and classes involved. MUD DATA HANDLING ----------------- Data can also come from the mud. Mud data comes from the net module's "SocketCommunicator" and is encapsulated in an "event.MudEvent". tbd. THE UI ------ All user interfaces extend the "ui.base.BaseUI" class. It has a series of methods which need to be overridden to give the ui functionality. In addition, ui's can specify additional Lyntin commands, they can bind to Lyntin hooks (most ui's bind to the "startup_hook" allowing them to perform certain functionality at Lyntin startup rather than when the module is imported or when the ui is instantiated), and anything else that modules can do. Lyntin comes with a text ui which can be used over telnet/ssh or in a command shell. It also comes with a tk gui. The tk gui has functionality like split history screen (pgup and pgdown), its own command history buffer (up and down) and numpad bindings (VK__NUMPAD0 through VK__NUMPAD9). Feel free to extend the ui code and send in patches. Or even write your own ui based on another widget set or whatever you want to do. For more information, read through the code in the ui package specifically the base module which defines the ui API. HOOKS ----- The engine is augmented by a series of hooks which allow modules to execute functions without having to change Lyntin's internals. Examples of this would be the "startup_hook" and "shutdown_hook". Any functions that hook into these hooks will be executed upon startup or shutdown of Lyntin. Lyntin also uses these hooks to implement its functionality. Plugins use hooks to hook into Lyntin and its processes. Hooks are defined throughout Lyntin by the various things that use the hook. Registering, unregistering, and retrieving Hooks should be done through the exported module. Read through the API documentation for hook documentation. CONFIGURATION ------------- Lyntin 4.0 has a sophisticated configuration system that's composed of boot options which are defined in the runlyntin startup script as well as provided by the config.ini file you can pass in at the command line. In-game, configuration can be adjusted using the #config command. The #config command will also tell you what configuration parameters are available, what the values are currently set at, and descriptions of what the configuration parameters do. As a module writer, you can add your own configuration parameters. See examples in the lyntin/modules/substitute.py module. Also see the documentation in the config module in the API as well as in the exported module. SOURCE CODE HACKING =================== Lyntin is entirely written in Python. The source code is well organized (this isn't really proven--it's more my opinion than anything else) and documentation exists on the web-site as well as in the source code distribution on how Lyntin is architected and details on the internals. Lyntin is distributed under the GPL. You should read this prior to making changes to the source code and distributing them so you don't accidentally find yourself in violation. If you patch Lyntin, we'd love to see your work! Send it to the Lyntin developers list and depending on what the patch does, it may get incorporated into the Lyntin base code. Note, you must disclaim your copyright on all code you submit so we can assign copyright to the FSF. If you don't, we won't accept it. LYNTIN MODULE HACKING ===================== Lyntin modules are cool and are what gives Lyntin much of its functionality. Because Lyntin modules are written in Python, they have full access to the Python API--they're not run in any kind of rexec bastion or anything. Every Lyntin module is written in Python. Lyntin modules can be loaded by Lyntin in two different ways. The first is at Lyntin startup when Lyntin lookes through all the Python modules in the Lyntin "modules/" directory excluding modules whose names start with a _. The second is using the "#import" Lyntin command: #import lyntinuser #import lyntin.modules.highlight When the Lyntin modules is loaded, Lyntin will check the module for a "load" function and if it exists, Lyntin will execute it. This function lets you separate Python importing of the module from the Lyntin initialization. All the Lyntin modules that come with Lyntin use the "load" function for binding to Lyntin hooks, adding new commands, adding new help text, and doing other initialization sorts of things. When using the "#import" command to load a module, it first checks to see if the module is already loaded. If it is and the module has an "unload" function it will call this prior to reloading the module and calling the "load" function if it exists. We typically use the "unload" function for unregistering functions with Lyntin hooks as well as removing commands. This allows you to cleanly reload modules you're testing/debugging without stopping and restarting your Lyntin session. Lyntin modules should use the "exported" module to access Lyntin internals. This API is guaranteed not to change (much) over versions of Lyntin. In the modules/ subdirectory, there's a module called modutils. This module holds some helper functions for making things a bit easier to deal with. Consult the documentation there for more guidance. Use the modules that come with Lyntin as a series of examples on writing your own modules. When you're writing your first module, be sure to brew a pot of coffee to aid you. WRITING MESSAGES TO THE UI -------------------------- In your modules, you should import the "exported" module. In there are a series of write_* functions which write various Message types to the user interface. For example: import exported def fancy_cmd(ses, args, input): text = args[0] if not text: exported.write_error("error: there is not text") return exported.write_message("text: %s" % text) See the documentation in the "exported" module for more details. ANSI colors are supported in the text you write to the ui. One way of doing this is as follows: exported.write_message("\033[1;34mhellothere\033[0m") However, that's difficult to read and subject to many potential tyops. As such, there's a get_color function in the "ansi" module which makes this a little easier: import ansi exported.write_message("%shello%s" % ansi.get_color("light blue"), ansi.get_color("default")) ANSI coloring is not a binary thing--there's no end tag. It's also not very nestable. Make sure to use "default" when ending your ANSI coloring--that returns the ANSI color to the default values for options, foreground, and background. COMMAND LINE HACKING ==================== Lyntin allows you to execute arbitrary Python at the command line. If you have a "lyntinuser" Lyntin module, then the code will be executed in there. Otherwise the code is executed in the "advanced" Lyntin module. For example: #@print "hello" will print hello to the console. There are some significant differences between Lyntin 2.x and 3.x in how this works. In the 2.x series, raw Python was executed in the "user" module. It was encouraged that you add your own functions to the "user" module and then these would be available to you at the Lyntin command line. In 3.0 and later versions, "lyntinuser" is a Lyntin module in the modules directory. It does not come with Lyntin--it's something you can create if you want to build your own environment to execute Python commands in. If you don't have a "lyntinuser" module, raw Python code gets executed in the "advanced" module where the #@ functionality exists. A basic "lyntinuser" module could be as follows:: """ This is my lyntinuser.py module. I can do whatever I want in here. I can store this module in my Lyntin moduledir (set at the command line using --moduledir ) or in Lyntin's modules subdirectory along with alias.py, action.py and the rest of the gang. """ import exported myvar = "hello" def myfunc(): exported.write_message(myvar) def load(): """ You can put code here for initializing your module when it's loaded here. Typically this is registering with Lyntin hooks and adding Lyntin commands. Look at the other Lyntin modules for examples. """ pass def unload(): """ Put code here for uninitializing your module when it's unloaded. Typically this is the reverse of the load module: unregistering functions with Lyntin hooks, remove Lyntin commands, removing Lyntin help topics... Look at other Lyntin modules for examples. """ pass Now at the Lyntin command prompt I can do things like:: > #@print myvar hello > #@myfunc() lyntin: hello THE END ======= That about covers it. Read through the API and the source code. If you have questions, subscribe to the lyntin-devl mailing list and ask us. http://sourceforge.net/mail/?group_id=6730 Happy hacking! the Lyntin development folks http://lyntin.sourceforge.net/