Update SERVER_JS_DEFINE_REGEX, so logging in on newer FB versions work

The locale is only used during error handling, and makes it harder for
users to report errors

.github/ISSUE_TEMPLATE/bug_report.md

@@ -21,8 +21,8 @@ Traceback (most recent call last):
   File "[site-packages]/fbchat/client.py", line 78, in __init__
     self.login(email, password, max_tries)
   File "[site-packages]/fbchat/client.py", line 407, in login
-    raise FBchatUserError('Login failed. Check email/password. (Failed on url: {})'.format(login_url))
+    raise FBchatException('Login failed. Check email/password. (Failed on URL: {})'.format(login_url))
-fbchat.models.FBchatUserError: Login failed. Check email/password. (Failed on url: https://m.facebook.com/login.php?login_attempt=1)
+fbchat.FBchatException: Login failed. Check email/password. (Failed on URL: https://m.facebook.com/login.php?login_attempt=1)
 ```
 
 ## Environment information

.gitignore

@@ -33,5 +33,9 @@ my_data.json
 tests.data
 .pytest_cache
 
+# MyPy
+.mypy_cache/
+
 # Virtual environment
 venv/
+.venv*/

.readthedocs.yml

@@ -0,0 +1,20 @@
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+version: 2
+
+formats:
+  - pdf
+  - htmlzip
+
+python:
+  version: 3.6
+  install:
+    - path: .
+      extra_requirements:
+        - docs
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  configuration: docs/conf.py
+  # Disabled, until we can find a way to get sphinx-autodoc-typehints play nice with our
+  # module renaming!
+  fail_on_warning: false

.travis.yml

@@ -7,21 +7,13 @@ cache: pip
 before_install: pip install flit
 # Use `--deps production` so that we don't install unnecessary dependencies
 install: flit install --deps production --extras test
-script: pytest -m offline
+script: pytest
 
 jobs:
   include:
-  - python: 2.7
-    before_install:
-    - sudo apt-get -y install python3-pip python3-setuptools
-    - sudo pip3 install flit
-    install: flit install --python python --deps production --extras test
-  - python: 3.4
   - python: 3.5
   - python: 3.6
   - python: 3.7
-    dist: xenial
-    sudo: required
   - python: pypy3.5
 
   - name: Lint
@@ -30,7 +22,7 @@ jobs:
     script: black --check --verbose .
 
   - stage: deploy
-    name: Github Releases
+    name: GitHub Releases
     if: tag IS present
     install: skip
     script: flit build

CONTRIBUTING.rst

@@ -1,38 +1,42 @@
-Contributing to fbchat
+Contributing to ``fbchat``
-======================
+==========================
 
 Thanks for reading this, all contributions are very much welcome!
 
-Please be aware that ``fbchat`` uses `Scemantic Versioning <https://semver.org/>`__
+Please be aware that ``fbchat`` uses `Scemantic Versioning <https://semver.org/>`__ quite rigorously!
 That means that if you're submitting a breaking change, it will probably take a while before it gets considered.
 
-In that case, you can point your PR to the ``2.0.0-dev`` branch, where the API is being properly developed.
-Otherwise, just point it to ``master``.
-
 Development Environment
 -----------------------
 
-You can use `flit` to install the package as a symlink:
+This project uses ``flit`` to configure development environments. You can install it using:
 
-.. code-block::
+.. code-block:: sh
 
+    $ pip install flit
+
+And now you can install ``fbchat`` as a symlink:
+
+.. code-block:: sh
+
+    $ git clone https://github.com/carpedm20/fbchat.git
+    $ cd fbchat
     $ # *nix:
     $ flit install --symlink
     $ # Windows:
     $ flit install --pth-file
 
+This will also install required development tools like ``black``, ``pytest`` and ``sphinx``.
+
 After that, you can ``import`` the module as normal.
 
-Before committing, you should run ``black .`` in the main directory, to format your code.
+Checklist
+---------
 
-Testing Environment
+Once you're done with your work, please follow the steps below:
--------------------
 
-The tests use `pytest <https://docs.pytest.org/>`__, and to work they need two Facebook accounts, and a group thread between these.
+- Run ``black .`` to format your code.
-To set these up, you should export the following environment variables:
+- Run ``pytest`` to test your code.
-
+- Run ``make -C docs html``, and view the generated docs, to verify that the docs still work.
-``client1_email``, ``client1_password``, ``client2_email``, ``client2_password`` and ``group_id``
+- Run ``make -C docs spelling`` to check your spelling in docstrings.
-
+- Create a pull request, and point it to ``master`` `here <https://github.com/carpedm20/fbchat/pulls/new>`__.
-If you're not able to do this, consider simply running ``pytest -m offline``.
-
-And if you're adding new functionality, if possible, make sure to create a new test for it.

README.rst

@@ -1,52 +1,112 @@
-fbchat: Facebook Chat (Messenger) for Python
+``fbchat`` - Facebook Messenger for Python
-============================================
+==========================================
 
-.. image:: https://img.shields.io/badge/license-BSD-blue.svg
+.. image:: https://badgen.net/pypi/v/fbchat
+    :target: https://pypi.python.org/pypi/fbchat
+    :alt: Project version
+
+.. image:: https://badgen.net/badge/python/3.5,3.6,3.7,3.8,pypy?list=|
+    :target: https://pypi.python.org/pypi/fbchat
+    :alt: Supported python versions: 3.5, 3.6, 3.7, 3.8 and pypy
+
+.. image:: https://badgen.net/pypi/license/fbchat
     :target: https://github.com/carpedm20/fbchat/tree/master/LICENSE
     :alt: License: BSD 3-Clause
 
-.. image:: https://img.shields.io/badge/python-2.7%2C%203.4%2C%203.5%2C%203.6%203.7%20pypy-blue.svg
+.. image:: https://readthedocs.org/projects/fbchat/badge/?version=stable
-    :target: https://pypi.python.org/pypi/fbchat
-    :alt: Supported python versions: 2.7, 3.4, 3.5, 3.6, 3.7 and pypy
-
-.. image:: https://readthedocs.org/projects/fbchat/badge/?version=master
     :target: https://fbchat.readthedocs.io
     :alt: Documentation
 
-.. image:: https://travis-ci.org/carpedm20/fbchat.svg?branch=master
+.. image:: https://badgen.net/travis/carpedm20/fbchat
     :target: https://travis-ci.org/carpedm20/fbchat
     :alt: Travis CI
 
-.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
+.. image:: https://badgen.net/badge/code%20style/black/black
     :target: https://github.com/ambv/black
     :alt: Code style
 
-Facebook Chat (`Messenger <https://www.facebook.com/messages/>`__) for Python.
+A powerful and efficient library to interact with
-This project was inspired by `facebook-chat-api <https://github.com/Schmavery/facebook-chat-api>`__.
+`Facebook's Messenger <https://www.facebook.com/messages/>`__, using just your email and password.
 
-**No XMPP or API key is needed**. Just use your email and password.
+This is *not* an official API, Facebook has that `over here <https://developers.facebook.com/docs/messenger-platform>`__ for chat bots. This library differs by using a normal Facebook account instead.
 
-Go to `Read the Docs <https://fbchat.readthedocs.io>`__ to see the full documentation,
+``fbchat`` currently support:
-or jump right into the code by viewing the `examples <https://github.com/carpedm20/fbchat/tree/master/examples>`__
 
-Installation:
+- Sending many types of messages, with files, stickers, mentions, etc.
+- Fetching all messages, threads and images in threads.
+- Searching for messages and threads.
+- Creating groups, setting the group emoji, changing nicknames, creating polls, etc.
+- Listening for, an reacting to messages and other events in real-time.
+- Type hints, and it has a modern codebase (e.g. only Python 3.5 and upwards).
+- ``async``/``await`` (COMING).
+
+Essentially, everything you need to make an amazing Facebook bot!
+
+
+Version Warning
+---------------
+``v2`` is currently being developed at the ``master`` branch and it's highly unstable. If you want to view the old ``v1``, go `here <https://github.com/carpedm20/fbchat/tree/v1>`__.
+
+Additionally, you can view the project's progress `here <https://github.com/carpedm20/fbchat/projects/2>`__.
+
+
+Caveats
+-------
+
+``fbchat`` works by imitating what the browser does, and thereby tricking Facebook into thinking it's accessing the website normally.
+
+However, there's a catch! **Using this library may not comply with Facebook's Terms Of Service!**, so be responsible Facebook citizens! We are not responsible if your account gets banned!
+
+Additionally, **the APIs the library is calling is undocumented!** In theory, this means that your code could break tomorrow, without the slightest warning!
+If this happens to you, please report it, so that we can fix it as soon as possible!
+
+.. inclusion-marker-intro-end
+.. This message doesn't make sense in the docs at Read The Docs, so we exclude it
+
+With that out of the way, you may go to `Read The Docs <https://fbchat.readthedocs.io/>`__ to see the full documentation!
+
+.. inclusion-marker-installation-start
+
+
+Installation
+------------
 
 .. code-block::
 
     $ pip install fbchat
 
-You can also install from source, by using `flit`:
+If you don't have `pip <https://pip.pypa.io/>`_, `this guide <http://docs.python-guide.org/en/latest/starting/installation/>`_ can guide you through the process.
+
+You can also install directly from source, provided you have ``pip>=19.0``:
 
 .. code-block::
 
-    $ pip install flit
+    $ pip install git+https://github.com/carpedm20/fbchat.git
-    $ git clone https://github.com/carpedm20/fbchat.git
+
-    $ cd fbchat
+.. inclusion-marker-installation-end
-    $ flit install
+
+
+Example Usage
+-------------
+
+.. code-block::
+
+    import getpass
+    import fbchat
+    session = fbchat.Session.login("<email/phone number>", getpass.getpass())
+    user = fbchat.User(session=session, id=session.user_id)
+    user.send_text("Test message!")
+
+More examples are available `here <https://github.com/carpedm20/fbchat/tree/master/examples>`__.
 
 
 Maintainer
 ----------
 
 - Mads Marquart / `@madsmtm <https://github.com/madsmtm>`__
-- Taehoon Kim / `@carpedm20 <http://carpedm20.github.io/about/>`__
+
+
+Acknowledgements
+----------------
+
+This project was originally inspired by `facebook-chat-api <https://github.com/Schmavery/facebook-chat-api>`__.

docs/Makefile

@@ -3,8 +3,7 @@
 
 # You can set these variables from the command line.
 SPHINXOPTS    =
-SPHINXBUILD   = python3.6 -msphinx
+SPHINXBUILD   = sphinx-build
-SPHINXPROJ    = fbchat
 SOURCEDIR     = .
 BUILDDIR      = _build
 

docs/_static/license.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="80" height="20"><linearGradient id="b" x2="0" y2="100%"><stop offset="0" stop-color="#bbb" stop-opacity=".1"/><stop offset="1" stop-opacity=".1"/></linearGradient><clipPath id="a"><rect width="80" height="20" rx="3" fill="#fff"/></clipPath><g clip-path="url(#a)"><path fill="#555" d="M0 0h47v20H0z"/><path fill="#007ec6" d="M47 0h33v20H47z"/><path fill="url(#b)" d="M0 0h80v20H0z"/></g><g fill="#fff" text-anchor="middle" font-family="DejaVu Sans,Verdana,Geneva,sans-serif" font-size="11"><text x="23.5" y="15" fill="#010101" fill-opacity=".3">license</text><text x="23.5" y="14">license</text><text x="62.5" y="15" fill="#010101" fill-opacity=".3">BSD</text><text x="62.5" y="14">BSD</text></g></svg>

docs/_static/python-versions.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="154" height="20"><linearGradient id="b" x2="0" y2="100%"><stop offset="0" stop-color="#bbb" stop-opacity=".1"/><stop offset="1" stop-opacity=".1"/></linearGradient><clipPath id="a"><rect width="154" height="20" rx="3" fill="#fff"/></clipPath><g clip-path="url(#a)"><path fill="#555" d="M0 0h49v20H0z"/><path fill="#007ec6" d="M49 0h105v20H49z"/><path fill="url(#b)" d="M0 0h154v20H0z"/></g><g fill="#fff" text-anchor="middle" font-family="DejaVu Sans,Verdana,Geneva,sans-serif" font-size="11"><text x="24.5" y="15" fill="#010101" fill-opacity=".3">python</text><text x="24.5" y="14">python</text><text x="100.5" y="15" fill="#010101" fill-opacity=".3">2.7, 3.4, 3.5, 3.6</text><text x="100.5" y="14">2.7, 3.4, 3.5, 3.6</text></g></svg>

docs/api.rst

@@ -1,44 +0,0 @@
-.. module:: fbchat
-.. highlight:: python
-.. _api:
-
-Full API
-========
-
-If you are looking for information on a specific function, class, or method, this part of the documentation is for you.
-
-
-.. _api_client:
-
-Client
-------
-
-This is the main class of `fbchat`, which contains all the methods you use to interact with Facebook.
-You can extend this class, and overwrite the events, to provide custom event handling (mainly used while listening)
-
-.. autoclass:: Client(email, password, user_agent=None, max_tries=5, session_cookies=None, logging_level=logging.INFO)
-    :members:
-
-
-.. _api_models:
-
-Models
-------
-
-These models are used in various functions, both as inputs and return values.
-A good tip is to write ``from fbchat.models import *`` at the start of your source, so you can use these models freely
-
-.. automodule:: fbchat.models
-    :members:
-    :undoc-members:
-
-
-.. _api_utils:
-
-Utils
------
-
-These functions and values are used internally by fbchat, and are subject to change. Do **NOT** rely on these to be backwards compatible!
-
-.. automodule:: fbchat.utils
-    :members:

docs/api/attachments.rst

@@ -0,0 +1,13 @@
+Attachments
+===========
+
+.. autoclass:: Attachment()
+.. autoclass:: ShareAttachment()
+.. autoclass:: Sticker()
+.. autoclass:: LocationAttachment()
+.. autoclass:: LiveLocationAttachment()
+.. autoclass:: FileAttachment()
+.. autoclass:: AudioAttachment()
+.. autoclass:: ImageAttachment()
+.. autoclass:: VideoAttachment()
+.. autoclass:: ImageAttachment()

docs/api/client.rst

@@ -0,0 +1,4 @@
+Client
+======
+
+.. autoclass:: Client

docs/api/events.rst

@@ -0,0 +1,4 @@
+Events
+======
+
+.. autoclass:: Listener

docs/api/exceptions.rst

@@ -0,0 +1,11 @@
+Exceptions
+==========
+
+.. autoexception:: FacebookError()
+.. autoexception:: HTTPError()
+.. autoexception:: ParseError()
+.. autoexception:: NotLoggedIn()
+.. autoexception:: ExternalError()
+.. autoexception:: GraphQLError()
+.. autoexception:: InvalidParameters()
+.. autoexception:: PleaseRefresh()

docs/api/index.rst

@@ -0,0 +1,21 @@
+.. module:: fbchat
+
+.. Note: we're using () to hide the __init__ method where relevant
+
+Full API
+========
+
+If you are looking for information on a specific function, class, or method, this part of the documentation is for you.
+
+.. toctree::
+    :maxdepth: 1
+
+    session
+    client
+    threads
+    thread_data
+    messages
+    exceptions
+    attachments
+    events
+    misc

docs/api/messages.rst

@@ -0,0 +1,8 @@
+Messages
+========
+
+.. autoclass:: Message
+.. autoclass:: Mention
+.. autoclass:: EmojiSize(Enum)
+    :undoc-members:
+.. autoclass:: MessageData()

docs/api/misc.rst

@@ -0,0 +1,20 @@
+Miscellaneous
+=============
+
+.. autoclass:: ThreadLocation(Enum)
+    :undoc-members:
+.. autoclass:: ActiveStatus()
+
+.. autoclass:: QuickReply
+.. autoclass:: QuickReplyText
+.. autoclass:: QuickReplyLocation
+.. autoclass:: QuickReplyPhoneNumber
+.. autoclass:: QuickReplyEmail
+
+.. autoclass:: Poll
+.. autoclass:: PollOption
+
+.. autoclass:: Plan
+.. autoclass:: PlanData()
+.. autoclass:: GuestStatus(Enum)
+    :undoc-members:

docs/api/session.rst

@@ -0,0 +1,4 @@
+Session
+=======
+
+.. autoclass:: Session()

docs/api/thread_data.rst

@@ -0,0 +1,6 @@
+Thread Data
+===========
+
+.. autoclass:: PageData()
+.. autoclass:: UserData()
+.. autoclass:: GroupData()

docs/api/threads.rst

@@ -0,0 +1,8 @@
+Threads
+=======
+
+.. autoclass:: ThreadABC()
+.. autoclass:: Thread
+.. autoclass:: Page
+.. autoclass:: User
+.. autoclass:: Group

docs/conf.py

@@ -1,37 +1,40 @@
-#!/usr/bin/env python3
+# Configuration file for the Sphinx documentation builder.
-# -*- coding: utf-8 -*-
 #
-# fbchat documentation build configuration file, created by
+# This file does only contain a selection of the most common options. For a
-# sphinx-quickstart on Thu May 25 15:43:01 2017.
+# full list see the documentation:
-#
+# http://www.sphinx-doc.org/en/master/config
-# This file is execfile()d with the current directory set to its
-# containing dir.
-#
-# Note that not all possible configuration values are present in this
-# autogenerated file.
-#
-# All configuration values have a default; values that are commented out
-# serve to show the default.
 
-# If extensions (or modules to document with autodoc) are in another directory,
+# -- Path setup --------------------------------------------------------------
-# add these directories to sys.path here. If the directory is relative to the
-# documentation root, use os.path.abspath to make it absolute, like shown here.
 
 import os
 import sys
 
 sys.path.insert(0, os.path.abspath(".."))
 
+os.environ["_FBCHAT_DISABLE_FIX_MODULE_METADATA"] = "1"
+
 import fbchat
-import tests
+
-from fbchat import __copyright__, __author__, __version__, __description__
+del os.environ["_FBCHAT_DISABLE_FIX_MODULE_METADATA"]
+
+# -- Project information -----------------------------------------------------
+
+project = fbchat.__name__
+copyright = "Copyright 2015 - 2018 by Taehoon Kim and 2018 - 2020 by Mads Marquart"
+author = "Taehoon Kim; Moreels Pieter-Jan; Mads Marquart"
+description = fbchat.__doc__.split("\n")[0]
+
+# The short X.Y version
+version = fbchat.__version__
+# The full version, including alpha/beta/rc tags
+release = fbchat.__version__
 
 
-# -- General configuration ------------------------------------------------
+# -- General configuration ---------------------------------------------------
 
 # If your documentation needs a minimal Sphinx version, state it here.
 #
-# needs_sphinx = '1.0'
+needs_sphinx = "2.0"
 
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
@@ -39,129 +42,54 @@ from fbchat import __copyright__, __author__, __version__, __description__
 extensions = [
     "sphinx.ext.autodoc",
     "sphinx.ext.intersphinx",
-    "sphinx.ext.todo",
     "sphinx.ext.viewcode",
+    "sphinx.ext.napoleon",
+    "sphinxcontrib.spelling",
+    "sphinx_autodoc_typehints",
 ]
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]
 
-# The suffix(es) of source filenames.
-# You can specify multiple suffix as a list of string:
-#
-# source_suffix = ['.rst', '.md']
-source_suffix = ".rst"
-
 # The master toctree document.
 master_doc = "index"
 
-# General information about the project.
-project = "fbchat"
-title = "fbchat Documentation"
-copyright = __copyright__
-author = __author__
-description = __description__
-
-# The version info for the project you're documenting, acts as replacement for
-# |version| and |release|, also used in various other places throughout the
-# built documents.
-#
-# The short X.Y version.
-version = __version__
-# The full version, including alpha/beta/rc tags.
-release = __version__
-
-# The language for content autogenerated by Sphinx. Refer to documentation
-# for a list of supported languages.
-#
-# This is also used if you do content translation via gettext catalogs.
-# Usually you set "language" from the command line for these cases.
-language = None
-
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
-# This patterns also effect to html_static_path and html_extra_path
+# This pattern also affects html_static_path and html_extra_path.
 exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
 
-# The name of the Pygments (syntax highlighting) style to use.
+rst_prolog = ".. currentmodule:: " + project
-pygments_style = "sphinx"
 
-# If true, `todo` and `todoList` produce output, else they produce nothing.
+# The reST default role (used for this markup: `text`) to use for all
-todo_include_todos = True
+# documents.
+#
+default_role = "any"
+
+# Make the reference parsing more strict
+#
+nitpicky = True
+
+# Prefer strict Python highlighting
+#
+highlight_language = "python3"
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#
+add_function_parentheses = False
 
 
-# -- Options for HTML output ----------------------------------------------
+# -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-
 html_theme = "alabaster"
 
 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the
 # documentation.
 #
-# html_theme_options = {}
-
-# Add any paths that contain custom static files (such as style sheets) here,
-# relative to this directory. They are copied after the builtin static files,
-# so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ["_static"]
-
-
-# -- Options for HTMLHelp output ------------------------------------------
-
-# Output file base name for HTML help builder.
-htmlhelp_basename = project + "doc"
-
-
-# -- Options for LaTeX output ---------------------------------------------
-
-latex_elements = {
-    # The paper size ('letterpaper' or 'a4paper').
-    #
-    # 'papersize': 'letterpaper',
-    # The font size ('10pt', '11pt' or '12pt').
-    #
-    # 'pointsize': '10pt',
-    # Additional stuff for the LaTeX preamble.
-    #
-    # 'preamble': '',
-    # Latex figure (float) alignment
-    #
-    # 'figure_align': 'htbp',
-}
-
-# Grouping the document tree into LaTeX files. List of tuples
-# (source start file, target name, title,
-#  author, documentclass [howto, manual, or own class]).
-latex_documents = [(master_doc, project + ".tex", title, author, "manual")]
-
-
-# -- Options for manual page output ---------------------------------------
-
-# One entry per manual page. List of tuples
-# (source start file, name, description, authors, manual section).
-man_pages = [(master_doc, project, title, [author], 1)]
-
-
-# -- Options for Texinfo output -------------------------------------------
-
-# Grouping the document tree into Texinfo files. List of tuples
-# (source start file, target name, title, author,
-#  dir menu entry, description, category)
-texinfo_documents = [
-    (master_doc, project, title, author, project, description, "Miscellaneous")
-]
-
-
-# Example configuration for intersphinx: refer to the Python standard library.
-intersphinx_mapping = {"https://docs.python.org/3/": None}
-
-
-add_function_parentheses = False
-
 html_theme_options = {
     "show_powered_by": False,
     "github_user": "carpedm20",
@@ -170,9 +98,97 @@ html_theme_options = {
     "show_related": False,
 }
 
+# Custom sidebar templates, must be a dictionary that maps document names
+# to template names.
+#
+# The default sidebars (for documents that don't match any pattern) are
+# defined by theme itself.  Builtin themes are using these templates by
+# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
+# 'searchbox.html']``.
+#
 html_sidebars = {"**": ["sidebar.html", "searchbox.html"]}
 
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#
 html_show_sphinx = False
+
+# If true, links to the reST sources are added to the pages.
+#
 html_show_sourcelink = False
-autoclass_content = "both"
+
+# A shorter title for the navigation bar. Default is the same as html_title.
+#
 html_short_title = description
+
+
+# -- Options for HTMLHelp output ---------------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = project + "doc"
+
+
+# -- Options for LaTeX output ------------------------------------------------
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [(master_doc, project + ".tex", project, author, "manual")]
+
+
+# -- Options for manual page output ------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [(master_doc, project, project, [x.strip() for x in author.split(";")], 1)]
+
+
+# -- Options for Texinfo output ----------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (master_doc, project, project, author, project, description, "Miscellaneous",)
+]
+
+
+# -- Options for Epub output -------------------------------------------------
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files = ["search.html"]
+
+
+# -- Extension configuration -------------------------------------------------
+
+# -- Options for autodoc extension ---------------------------------------
+
+autoclass_content = "class"
+autodoc_member_order = "bysource"
+autodoc_default_options = {"members": True}
+
+# -- Options for intersphinx extension ---------------------------------------
+
+# Example configuration for intersphinx: refer to the Python standard library.
+intersphinx_mapping = {"https://docs.python.org/": None}
+
+# -- Options for napoleon extension ----------------------------------------------
+
+# Use Google style docstrings
+napoleon_google_docstring = True
+napoleon_numpy_docstring = False
+
+# napoleon_use_admonition_for_examples = False
+# napoleon_use_admonition_for_notes = False
+# napoleon_use_admonition_for_references = False
+
+# -- Options for spelling extension ----------------------------------------------
+
+spelling_word_list_filename = [
+    "spelling/names.txt",
+    "spelling/technical.txt",
+    "spelling/fixes.txt",
+]
+spelling_ignore_wiki_words = False
+# spelling_ignore_acronyms = False
+spelling_ignore_python_builtins = False
+spelling_ignore_importable_modules = False

docs/examples.rst

@@ -1,16 +1,15 @@
-.. highlight:: python
 .. _examples:
 
 Examples
 ========
 
-These are a few examples on how to use `fbchat`. Remember to swap out `<email>` and `<password>` for your email and password
+These are a few examples on how to use ``fbchat``. Remember to swap out ``<email>`` and ``<password>`` for your email and password
 
 
 Basic example
 -------------
 
-This will show basic usage of `fbchat`
+This will show basic usage of ``fbchat``
 
 .. literalinclude:: ../examples/basic_usage.py
 
@@ -18,7 +17,7 @@ This will show basic usage of `fbchat`
 Interacting with Threads
 ------------------------
 
-This will interact with the thread in every way `fbchat` supports
+This will interact with the thread in every way ``fbchat`` supports
 
 .. literalinclude:: ../examples/interract.py
 
@@ -31,8 +30,8 @@ This will show the different ways of fetching information about users and thread
 .. literalinclude:: ../examples/fetch.py
 
 
-Echobot
+``Echobot``
--------
+-----------
 
 This will reply to any message with the same message
 
@@ -42,7 +41,7 @@ This will reply to any message with the same message
 Remove Bot
 ----------
 
-This will remove a user from a group if they write the message `Remove me!`
+This will remove a user from a group if they write the message ``Remove me!``
 
 .. literalinclude:: ../examples/removebot.py
 

docs/faq.rst

@@ -1,44 +1,23 @@
-.. highlight:: python
+Frequently Asked Questions
-.. module:: fbchat
+==========================
-.. _faq:
 
-FAQ
+The new version broke my application
-===
+------------------------------------
 
-Version X broke my installation
+``fbchat`` follows `Scemantic Versioning <https://semver.org/>`__ quite rigorously!
--------------------------------
 
-We try to provide backwards compatibility where possible, but since we're not part of Facebook,
+That means that breaking changes can *only* occur in major versions (e.g. ``v1.9.6`` -> ``v2.0.0``).
-most of the things may be broken at any point in time
 
-Downgrade to an earlier version of fbchat, run this command
+If you find that something breaks, and you didn't update to a new major version, then it is a bug, and we would be grateful if you reported it!
+
+In case you're stuck with an old codebase, you can downgrade to a previous version of ``fbchat``, e.g. version ``1.9.6``:
 
 .. code-block:: sh
 
-    $ pip install fbchat==<X>
+    $ pip install fbchat==1.9.6
-
-Where you replace ``<X>`` with the version you want to use
 
 
 Will you be supporting creating posts/events/pages and so on?
 -------------------------------------------------------------
 
-We won't be focusing on anything else than chat-related things. This API is called `fbCHAT`, after all ;)
+We won't be focusing on anything else than chat-related things. This library is called ``fbCHAT``, after all!
-
-
-Submitting Issues
------------------
-
-If you're having trouble with some of the snippets, or you think some of the functionality is broken,
-please feel free to submit an issue on `Github <https://github.com/carpedm20/fbchat>`_.
-You should first login with ``logging_level`` set to ``logging.DEBUG``::
-
-    from fbchat import Client
-    import logging
-    client = Client('<email>', '<password>', logging_level=logging.DEBUG)
-
-Then you can submit the relevant parts of this log, and detailed steps on how to reproduce
-
-.. warning::
-    Always remove your credentials from any debug information you may provide us.
-    Preferably, use a test account, in case you miss anything

docs/index.rst

@@ -1,66 +1,23 @@
-.. highlight:: python
+.. highlight:: sh
-.. module:: fbchat
+.. See README.rst for explanation of these markers
-.. fbchat documentation master file, created by
-   sphinx-quickstart on Thu May 25 15:43:01 2017.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
 
-.. This documentation's layout is heavily inspired by requests' layout: https://requests.readthedocs.io
+.. include:: ../README.rst
-   Some documentation is also partially copied from facebook-chat-api: https://github.com/Schmavery/facebook-chat-api
+    :end-before: inclusion-marker-intro-end
 
-fbchat: Facebook Chat (Messenger) for Python
+With that said, let's get started!
-============================================
 
-Release v\ |version|. (:ref:`install`)
+.. include:: ../README.rst
-
+    :start-after: inclusion-marker-installation-start
-.. generated with: https://img.shields.io/badge/license-BSD-blue.svg
+    :end-before: inclusion-marker-installation-end
-
-.. image:: /_static/license.svg
-    :target: https://github.com/carpedm20/fbchat/blob/master/LICENSE.txt
-    :alt: License: BSD
-
-.. generated with: https://img.shields.io/badge/python-2.7%2C%203.4%2C%203.5%2C%203.6-blue.svg
-
-.. image:: /_static/python-versions.svg
-    :target: https://pypi.python.org/pypi/fbchat
-    :alt: Supported python versions: 2.7, 3.4, 3.5 and 3.6
-
-Facebook Chat (`Messenger <https://www.facebook.com/messages/>`_) for Python.
-This project was inspired by `facebook-chat-api <https://github.com/Schmavery/facebook-chat-api>`_.
-
-**No XMPP or API key is needed**. Just use your email and password.
-
-Currently `fbchat` support Python 2.7, 3.4, 3.5 and 3.6:
-
-`fbchat` works by emulating the browser.
-This means doing the exact same GET/POST requests and tricking Facebook into thinking it's accessing the website normally.
-Therefore, this API requires the credentials of a Facebook account.
-
-.. note::
-    If you're having problems, please check the :ref:`faq`, before asking questions on Github
-
-.. warning::
-    We are not responsible if your account gets banned for spammy activities,
-    such as sending lots of messages to people you don't know, sending messages very quickly,
-    sending spammy looking URLs, logging in and out very quickly... Be responsible Facebook citizens.
-
-.. note::
-    Facebook now has an `official API <https://developers.facebook.com/docs/messenger-platform>`_ for chat bots,
-    so if you're familiar with node.js, this might be what you're looking for.
-
-If you're already familiar with the basics of how Facebook works internally, go to :ref:`examples` to see example usage of `fbchat`
 
 
-Overview
+Documentation Overview
---------
+----------------------
 
 .. toctree::
     :maxdepth: 2
 
-    install
     intro
     examples
-    testing
-    api
-    todo
     faq
+    api/index

docs/install.rst

@@ -1,36 +0,0 @@
-.. highlight:: sh
-.. _install:
-
-Installation
-============
-
-Pip Install fbchat
-------------------
-
-To install fbchat, run this command::
-
-    $ pip install fbchat
-
-If you don't have `pip <https://pip.pypa.io>`_ installed,
-`this Python installation guide <http://docs.python-guide.org/en/latest/starting/installation/>`_
-can guide you through the process.
-
-Get the Source Code
--------------------
-
-fbchat is developed on GitHub, where the code is
-`always available <https://github.com/carpedm20/fbchat>`_.
-
-You can either clone the public repository::
-
-    $ git clone git://github.com/carpedm20/fbchat.git
-
-Or, download a `tarball <https://github.com/carpedm20/fbchat/tarball/master>`_::
-
-    $ curl -OL https://github.com/carpedm20/fbchat/tarball/master
-      # optionally, zipball is also available (for Windows users).
-
-Once you have a copy of the source, you can embed it in your own Python
-package, or install it into your site-packages easily::
-
-    $ python setup.py install

docs/intro.rst

@@ -1,200 +1,152 @@
-.. highlight:: python
-.. module:: fbchat
-.. _intro:
-
 Introduction
 ============
 
-`fbchat` uses your email and password to communicate with the Facebook server.
+Welcome, this page will guide you through the basic concepts of using ``fbchat``.
-That means that you should always store your password in a separate file, in case e.g. someone looks over your shoulder while you're writing code.
-You should also make sure that the file's access control is appropriately restrictive
 
+The hardest, and most error prone part is logging in, and managing your login session, so that is what we will look at first.
 
-.. _intro_logging_in:
 
 Logging In
 ----------
 
-Simply create an instance of :class:`Client`. If you have two factor authentication enabled, type the code in the terminal prompt
+Everything in ``fbchat`` starts with getting an instance of `Session`. Currently there are two ways of doing that, `Session.login` and `Session.from_cookies`.
-(If you want to supply the code in another fashion, overwrite :func:`Client.on2FACode`)::
 
-    from fbchat import Client
+The follow example will prompt you for you password, and use it to login::
-    from fbchat.models import *
-    client = Client('<email>', '<password>')
 
-Replace ``<email>`` and ``<password>`` with your email and password respectively
+    import getpass
+    import fbchat
+    session = fbchat.Session.login("<email/phone number>", getpass.getpass())
+    # If your account requires a two factor authentication code:
+    session = fbchat.Session.login(
+        "<your email/phone number>",
+        getpass.getpass(),
+        lambda: getpass.getpass("2FA code"),
+    )
 
-.. note::
+However, **this is not something you should do often!** Logging in/out all the time *will* get your Facebook account locked!
-    For ease of use then most of the code snippets in this document will assume you've already completed the login process
-    Though the second line, ``from fbchat.models import *``, is not strictly neccesary here, later code snippets will assume you've done this
 
-If you want to change how verbose `fbchat` is, change the logging level (in :class:`Client`)
+Instead, you should start by using `Session.login`, and then store the cookies with `Session.get_cookies`, so that they can be used instead the next time your application starts.
 
-Throughout your code, if you want to check whether you are still logged in, use :func:`Client.isLoggedIn`.
+Usability-wise, this is also better, since you won't have to re-type your password every time you want to login.
-An example would be to login again if you've been logged out, using :func:`Client.login`::
 
-    if not client.isLoggedIn():
+The following, quite lengthy, yet very import example, illustrates a way to do this:
-        client.login('<email>', '<password>')
 
-When you're done using the client, and want to securely logout, use :func:`Client.logout`::
+.. literalinclude:: ../examples/session_handling.py
 
-    client.logout()
+Assuming you have successfully completed the above, congratulations! Using ``fbchat`` should be mostly trouble free from now on!
 
 
-.. _intro_threads:
+Understanding Thread Ids
+------------------------
 
-Threads
+At the core of any thread is its unique identifier, its ID.
--------
 
-A thread can refer to two things: A Messenger group chat or a single Facebook user
+A thread basically just means "something I can chat with", but more precisely, it can refer to a few things:
+- A Messenger group thread (`Group`)
+- The conversation between you and a single Facebook user (`User`)
+- The conversation between you and a Facebook Page (`Page`)
 
-:class:`models.ThreadType` is an enumerator with two values: ``USER`` and ``GROUP``.
+You can get your own user ID from `Session.user` with ``session.user.id``.
-These will specify whether the thread is a single user chat or a group chat.
-This is required for many of `fbchat`'s functions, since Facebook differentiates between these two internally
 
-Searching for group chats and finding their ID can be done via. :func:`Client.searchForGroups`,
+Getting the ID of a specific group thread is fairly trivial, you only need to login to `<https://www.messenger.com/>`_, click on the group you want to find the ID of, and then read the id from the address bar.
-and searching for users is possible via. :func:`Client.searchForUsers`. See :ref:`intro_fetching`
+The URL will look something like this: ``https://www.messenger.com/t/1234567890``, where ``1234567890`` would be the ID of the group.
 
-You can get your own user ID by using :any:`Client.uid`
+The same method can be applied to some user accounts, though if they have set a custom URL, then you will have to use a different method.
 
-Getting the ID of a group chat is fairly trivial otherwise, since you only need to navigate to `<https://www.facebook.com/messages/>`_,
+An image to illustrate the process is shown below:
-click on the group you want to find the ID of, and then read the id from the address bar.
-The URL will look something like this: ``https://www.facebook.com/messages/t/1234567890``, where ``1234567890`` would be the ID of the group.
-An image to illustrate this is shown below:
 
 .. image:: /_static/find-group-id.png
     :alt: An image illustrating how to find the ID of a group
 
-The same method can be applied to some user accounts, though if they've set a custom URL, then you'll just see that URL instead
+Once you have an ID, you can use it to create a `Group` or a `User` instance, which will allow you to do all sorts of things. To do this, you need a valid, logged in session::
 
-Here's an snippet showing the usage of thread IDs and thread types, where ``<user id>`` and ``<group id>``
+    group = fbchat.Group(session=session, id="<The id you found>")
-corresponds to the ID of a single user, and the ID of a group respectively::
+    # Or for user threads
+    user = fbchat.User(session=session, id="<The id you found>")
 
-    client.send(Message(text='<message>'), thread_id='<user id>', thread_type=ThreadType.USER)
+Just like threads, every message, poll, plan, attachment, action etc. you send or do on Facebook has a unique ID.
-    client.send(Message(text='<message>'), thread_id='<group id>', thread_type=ThreadType.GROUP)
 
-Some functions (e.g. :func:`Client.changeThreadColor`) don't require a thread type, so in these cases you just provide the thread ID::
+Below is an example of using such a message ID to get a `Message` instance::
 
-    client.changeThreadColor(ThreadColor.BILOBA_FLOWER, thread_id='<user id>')
+    # Provide the thread the message was created in, and it's ID
-    client.changeThreadColor(ThreadColor.MESSENGER_BLUE, thread_id='<group id>')
+    message = fbchat.Message(thread=user, id="<The message id>")
 
 
-.. _intro_message_ids:
-
-Message IDs
------------
-
-Every message you send on Facebook has a unique ID, and every action you do in a thread,
-like changing a nickname or adding a person, has a unique ID too.
-
-Some of `fbchat`'s functions require these ID's, like :func:`Client.reactToMessage`,
-and some of then provide this ID, like :func:`Client.sendMessage`.
-This snippet shows how to send a message, and then use the returned ID to react to that message with a 😍 emoji::
-
-    message_id = client.send(Message(text='message'), thread_id=thread_id, thread_type=thread_type)
-    client.reactToMessage(message_id, MessageReaction.LOVE)
-
-
-.. _intro_interacting:
-
-Interacting with Threads
-------------------------
-
-`fbchat` provides multiple functions for interacting with threads
-
-Most functionality works on all threads, though some things,
-like adding users to and removing users from a group chat, logically only works on group chats
-
-The simplest way of using `fbchat` is to send a message.
-The following snippet will, as you've probably already figured out, send the message `test message` to your account::
-
-    message_id = client.send(Message(text='test message'), thread_id=client.uid, thread_type=ThreadType.USER)
-
-You can see a full example showing all the possible thread interactions with `fbchat` by going to :ref:`examples`
-
-
-.. _intro_fetching:
-
 Fetching Information
 --------------------
 
-You can use `fbchat` to fetch basic information like user names, profile pictures, thread names and user IDs
+Managing these ids yourself quickly becomes very cumbersome! Luckily, there are other, easier ways of getting `Group`/`User` instances.
 
-You can retrieve a user's ID with :func:`Client.searchForUsers`.
+You would start by creating a `Client` instance, which is basically just a helper on top of `Session`, that will allow you to do various things::
-The following snippet will search for users by their name, take the first (and most likely) user, and then get their user ID from the result::
 
-    users = client.searchForUsers('<name of user>')
+    client = fbchat.Client(session=session)
-    user = users[0]
-    print("User's ID: {}".format(user.uid))
-    print("User's name: {}".format(user.name))
-    print("User's profile picture url: {}".format(user.photo))
-    print("User's main url: {}".format(user.url))
 
-Since this uses Facebook's search functions, you don't have to specify the whole name, first names will usually be enough
+Now, you could search for threads using `Client.search_for_threads`, or fetch a list of them using `Client.fetch_threads`::
 
-You can see a full example showing all the possible ways to fetch information with `fbchat` by going to :ref:`examples`
+    # Fetch the 5 most likely search results
+    # Uses Facebook's search functions, you don't have to specify the whole name, first names will usually be enough
+    threads = list(client.search_for_threads("<name of the thread to search for>", limit=5))
+    # Fetch the 5 most recent threads in your account
+    threads = list(client.fetch_threads(limit=5))
+
+Note the `list` statements; this is because the methods actually return `generators <https://wiki.python.org/moin/Generators>`__. If you don't know what that means, don't worry, it is just something you can use to make your code faster later.
+
+The examples above will actually fetch `UserData`/`GroupData`, which are subclasses of `User`/`Group`. These model have extra properties, so you could for example print the names and ids of the fetched threads like this::
+
+    for thread in threads:
+        print(f"{thread.id}: {thread.name}")
+
+Once you have a thread, you can use that to fetch the messages therein::
+
+    for message in thread.fetch_messages(limit=20):
+        print(message.text)
 
 
-.. _intro_sessions:
+Interacting with Threads
+------------------------
 
-Sessions
+Once you have a `User`/`Group` instance, you can do things on them as described in `ThreadABC`, since they are subclasses of that.
---------
 
-`fbchat` provides functions to retrieve and set the session cookies.
+Some functionality, like adding users to a `Group`, or blocking a `User`, logically only works the relevant threads, so see the full API documentation for that.
-This will enable you to store the session cookies in a separate file, so that you don't have to login each time you start your script.
-Use :func:`Client.getSession` to retrieve the cookies::
 
-    session_cookies = client.getSession()
+With that out of the way, let's see some examples!
 
-Then you can use :func:`Client.setSession`::
+The simplest way of interacting with a thread is by sending a message::
 
-    client.setSession(session_cookies)
+    # Send a message to the user
+    message = user.send_text("test message")
 
-Or you can set the ``session_cookies`` on your initial login.
+There are many types of messages you can send, see the full API documentation for more.
-(If the session cookies are invalid, your email and password will be used to login instead)::
 
-    client = Client('<email>', '<password>', session_cookies=session_cookies)
+Notice how we held on to the sent message? The return type i a `Message` instance, so you can interact with it afterwards::
 
-.. warning::
+    # React to the message with the 😍 emoji
-    You session cookies can be just as valueable as you password, so store them with equal care
+    message.react("😍")
 
+Besides sending messages, you can also interact with threads in other ways. An example is to change the thread color::
+
+    # Will change the thread color to the default blue
+    thread.set_color("#0084ff")
 
-.. _intro_events:
 
 Listening & Events
 ------------------
 
-To use the listening functions `fbchat` offers (like :func:`Client.listen`),
+Now, we are finally at the point we have all been waiting for: Creating an automatic Facebook bot!
-you have to define what should be executed when certain events happen.
-By default, (most) events will just be a `logging.info` statement,
-meaning it will simply print information to the console when an event happens
 
-.. note::
+To get started, you create the functions you want to call on certain events::
-    You can identify the event methods by their `on` prefix, e.g. `onMessage`
 
-The event actions can be changed by subclassing the :class:`Client`, and then overwriting the event methods::
+    def my_function(event: fbchat.MessageEvent):
+        print(f"Message from {event.author.id}: {event.message.text}")
 
-    class CustomClient(Client):
+Then you create a `fbchat.Listener` object::
-        def onMessage(self, mid, author_id, message_object, thread_id, thread_type, ts, metadata, msg, **kwargs):
-            # Do something with message_object here
-            pass
 
-    client = CustomClient('<email>', '<password>')
+    listener = fbchat.Listener(session=session, chat_on=False, foreground=False)
 
-**Notice:** The following snippet is as equally valid as the previous one::
+Which you can then use to receive events, and send them to your functions::
 
-    class CustomClient(Client):
+    for event in listener.listen():
-        def onMessage(self, message_object, author_id, thread_id, thread_type, **kwargs):
+        if isinstance(event, fbchat.MessageEvent):
-            # Do something with message_object here
+            my_function(event)
-            pass
 
-    client = CustomClient('<email>', '<password>')
+View the :ref:`examples` to see some more examples illustrating the event system.
-
-The change was in the parameters that our `onMessage` method took: ``message_object`` and ``author_id`` got swapped,
-and ``mid``, ``ts``, ``metadata`` and ``msg`` got removed, but the function still works, since we included ``**kwargs``
-
-.. note::
-    Therefore, for both backwards and forwards compatability,
-    the API actually requires that you include ``**kwargs`` as your final argument.
-
-View the :ref:`examples` to see some more examples illustrating the event system

docs/make.bat

@@ -5,21 +5,20 @@ pushd %~dp0
 REM Command file for Sphinx documentation
 
 if "%SPHINXBUILD%" == "" (
-	set SPHINXBUILD=python -msphinx
+	set SPHINXBUILD=sphinx-build
 )
 set SOURCEDIR=.
 set BUILDDIR=_build
-set SPHINXPROJ=fbchat
 
 if "%1" == "" goto help
 
 %SPHINXBUILD% >NUL 2>NUL
 if errorlevel 9009 (
 	echo.
-	echo.The Sphinx module was not found. Make sure you have Sphinx installed,
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
-	echo.then set the SPHINXBUILD environment variable to point to the full
+	echo.installed, then set the SPHINXBUILD environment variable to point
-	echo.path of the 'sphinx-build' executable. Alternatively you may add the
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
-	echo.Sphinx directory to PATH.
+	echo.may add the Sphinx directory to PATH.
 	echo.
 	echo.If you don't have Sphinx installed, grab it from
 	echo.http://sphinx-doc.org/

docs/spelling/fixes.txt

@@ -0,0 +1,3 @@
+premade
+todo
+emoji

docs/spelling/names.txt

@@ -0,0 +1,3 @@
+Facebook
+GraphQL
+GitHub

docs/spelling/technical.txt

@@ -0,0 +1,17 @@
+iterables
+iterable
+mimetype
+timestamp
+metadata
+spam
+spammy
+admin
+admins
+unsend
+unsends
+unmute
+spritemap
+online
+inbox
+subclassing
+codebase

docs/testing.rst

@@ -1,29 +0,0 @@
-.. highlight:: sh
-.. module:: fbchat
-.. _testing:
-
-Testing
-=======
-
-To use the tests, copy ``tests/data.json`` to ``tests/my_data.json`` or type the information manually in the terminal prompts.
-
-- email: Your (or a test user's) email / phone number
-- password: Your (or a test user's) password
-- group_thread_id: A test group that will be used to test group functionality
-- user_thread_id: A person that will be used to test kick/add functionality (This user should be in the group)
-
-Please remember to test all supported python versions.
-If you've made any changes to the 2FA functionality, test it with a 2FA enabled account.
-
-If you only want to execute specific tests, pass the function names in the command line (not including the `test_` prefix). Example::
-
-    $ python tests.py sendMessage sessions sendEmoji
-
-.. warning::
-
-    Do not execute the full set of tests in too quick succession. This can get your account temporarily blocked for spam!
-    (You should execute the script at max about 10 times a day)
-
-.. automodule:: tests
-    :members: TestFbchat
-    :undoc-members: TestFbchat

docs/todo.rst

@@ -1,24 +0,0 @@
-.. highlight:: python
-.. module:: fbchat
-.. _todo:
-
-Todo
-====
-
-This page will be periodically updated to show missing features and documentation
-
-
-Missing Functionality
----------------------
-
-- Implement Client.searchForMessage
-    - This will use the graphql request API
-- Implement chatting with pages properly
-- Write better FAQ
-- Explain usage of graphql
-
-
-Documentation
--------------
-
-.. todolist::

examples/basic_usage.py

@@ -1,12 +1,12 @@
-# -*- coding: UTF-8 -*-
+import fbchat
 
-from fbchat import Client
+# Log the user in
-from fbchat.models import *
+session = fbchat.Session.login("<email>", "<password>")
 
-client = Client("<email>", "<password>")
+print("Own id: {}".format(session.user.id))
 
-print("Own id: {}".format(client.uid))
+# Send a message to yourself
+session.user.send_text("Hi me!")
 
-client.send(Message(text="Hi me!"), thread_id=client.uid, thread_type=ThreadType.USER)
+# Log the user out
-
+session.logout()
-client.logout()

examples/echobot.py

@@ -1,19 +1,11 @@
-# -*- coding: UTF-8 -*-
+import fbchat
 
-from fbchat import log, Client
+session = fbchat.Session.login("<email>", "<password>")
-
+listener = fbchat.Listener(session=session, chat_on=False, foreground=False)
-# Subclass fbchat.Client and override required methods
-class EchoBot(Client):
-    def onMessage(self, author_id, message_object, thread_id, thread_type, **kwargs):
-        self.markAsDelivered(thread_id, message_object.uid)
-        self.markAsRead(thread_id)
-
-        log.info("{} from {} in {}".format(message_object, thread_id, thread_type.name))
 
+for event in listener.listen():
+    if isinstance(event, fbchat.MessageEvent):
+        print(f"{event.message.text} from {event.author.id} in {event.thread.id}")
         # If you're not the author, echo
-        if author_id != self.uid:
+        if event.author.id != session.user.id:
-            self.send(message_object, thread_id=thread_id, thread_type=thread_type)
+            event.thread.send_text(event.message.text)
-
-
-client = EchoBot("<email>", "<password>")
-client.listen()

examples/fetch.py

@@ -1,46 +1,50 @@
-# -*- coding: UTF-8 -*-
+import fbchat
 
-from fbchat import Client
+session = fbchat.Session.login("<email>", "<password>")
-from fbchat.models import *
 
-client = Client("<email>", "<password>")
+client = fbchat.Client(session=session)
 
 # Fetches a list of all users you're currently chatting with, as `User` objects
-users = client.fetchAllUsers()
+users = client.fetch_all_users()
 
-print("users' IDs: {}".format([user.uid for user in users]))
+print("users' IDs: {}".format([user.id for user in users]))
 print("users' names: {}".format([user.name for user in users]))
 
 
-# If we have a user id, we can use `fetchUserInfo` to fetch a `User` object
+# If we have a user id, we can use `fetch_user_info` to fetch a `User` object
-user = client.fetchUserInfo("<user id>")["<user id>"]
+user = client.fetch_user_info("<user id>")["<user id>"]
 # We can also query both mutiple users together, which returns list of `User` objects
-users = client.fetchUserInfo("<1st user id>", "<2nd user id>", "<3rd user id>")
+users = client.fetch_user_info("<1st user id>", "<2nd user id>", "<3rd user id>")
 
 print("user's name: {}".format(user.name))
 print("users' names: {}".format([users[k].name for k in users]))
 
 
-# `searchForUsers` searches for the user and gives us a list of the results,
+# `search_for_users` searches for the user and gives us a list of the results,
 # and then we just take the first one, aka. the most likely one:
-user = client.searchForUsers("<name of user>")[0]
+user = client.search_for_users("<name of user>")[0]
 
-print("user ID: {}".format(user.uid))
+print("user ID: {}".format(user.id))
 print("user's name: {}".format(user.name))
 print("user's photo: {}".format(user.photo))
 print("Is user client's friend: {}".format(user.is_friend))
 
 
 # Fetches a list of the 20 top threads you're currently chatting with
-threads = client.fetchThreadList()
+threads = client.fetch_thread_list()
 # Fetches the next 10 threads
-threads += client.fetchThreadList(offset=20, limit=10)
+threads += client.fetch_thread_list(offset=20, limit=10)
 
 print("Threads: {}".format(threads))
 
 
+# If we have a thread id, we can use `fetch_thread_info` to fetch a `Thread` object
+thread = client.fetch_thread_info("<thread id>")["<thread id>"]
+print("thread's name: {}".format(thread.name))
+
+
 # Gets the last 10 messages sent to the thread
-messages = client.fetchThreadMessages(thread_id="<thread id>", limit=10)
+messages = thread.fetch_messages(limit=10)
 # Since the message come in reversed order, reverse them
 messages.reverse()
 
@@ -49,16 +53,17 @@ for message in messages:
     print(message.text)
 
 
-# If we have a thread id, we can use `fetchThreadInfo` to fetch a `Thread` object
+# `search_for_threads` searches works like `search_for_users`, but gives us a list of threads instead
-thread = client.fetchThreadInfo("<thread id>")["<thread id>"]
+thread = client.search_for_threads("<name of thread>")[0]
 print("thread's name: {}".format(thread.name))
-print("thread's type: {}".format(thread.type))
-
-
-# `searchForThreads` searches works like `searchForUsers`, but gives us a list of threads instead
-thread = client.searchForThreads("<name of thread>")[0]
-print("thread's name: {}".format(thread.name))
-print("thread's type: {}".format(thread.type))
 
 
 # Here should be an example of `getUnread`
+
+
+# Print image url for up to 20 last images from thread.
+images = list(thread.fetch_images(limit=20))
+for image in images:
+    if isinstance(image, fbchat.ImageAttachment):
+        url = client.fetch_image_url(image.id)
+        print(url)

examples/interract.py

@@ -1,93 +1,66 @@
-# -*- coding: UTF-8 -*-
+import fbchat
+import requests
 
-from fbchat import Client
+session = fbchat.Session.login("<email>", "<password>")
-from fbchat.models import *
 
-client = Client("<email>", "<password>")
+client = fbchat.Client(session)
 
-thread_id = "1234567890"
+thread = session.user
-thread_type = ThreadType.GROUP
+# thread = fbchat.User(session=session, id="0987654321")
+# thread = fbchat.Group(session=session, id="1234567890")
 
 # Will send a message to the thread
-client.send(Message(text="<message>"), thread_id=thread_id, thread_type=thread_type)
+thread.send_text("<message>")
 
 # Will send the default `like` emoji
-client.send(
+thread.send_sticker(fbchat.EmojiSize.LARGE.value)
-    Message(emoji_size=EmojiSize.LARGE), thread_id=thread_id, thread_type=thread_type
-)
 
 # Will send the emoji `👍`
-client.send(
+thread.send_emoji("👍", size=fbchat.EmojiSize.LARGE)
-    Message(text="👍", emoji_size=EmojiSize.LARGE),
-    thread_id=thread_id,
-    thread_type=thread_type,
-)
 
 # Will send the sticker with ID `767334476626295`
-client.send(
+thread.send_sticker("767334476626295")
-    Message(sticker=Sticker("767334476626295")),
-    thread_id=thread_id,
-    thread_type=thread_type,
-)
 
 # Will send a message with a mention
-client.send(
+thread.send_text(
-    Message(
+    text="This is a @mention",
-        text="This is a @mention", mentions=[Mention(thread_id, offset=10, length=8)]
+    mentions=[fbchat.Mention(thread.id, offset=10, length=8)],
-    ),
-    thread_id=thread_id,
-    thread_type=thread_type,
 )
 
 # Will send the image located at `<image path>`
-client.sendLocalImage(
+with open("<image path>", "rb") as f:
-    "<image path>",
+    files = client.upload([("image_name.png", f, "image/png")])
-    message=Message(text="This is a local image"),
+thread.send_text(text="This is a local image", files=files)
-    thread_id=thread_id,
-    thread_type=thread_type,
-)
 
-# Will download the image at the url `<image url>`, and then send it
+# Will download the image at the URL `<image url>`, and then send it
-client.sendRemoteImage(
+r = requests.get("<image url>")
-    "<image url>",
+files = client.upload([("image_name.png", r.content, "image/png")])
-    message=Message(text="This is a remote image"),
+thread.send_files(files)  # Alternative to .send_text
-    thread_id=thread_id,
-    thread_type=thread_type,
-)
 
 
 # Only do these actions if the thread is a group
-if thread_type == ThreadType.GROUP:
+if isinstance(thread, fbchat.Group):
-    # Will remove the user with ID `<user id>` from the thread
+    # Will remove the user with ID `<user id>` from the group
-    client.removeUserFromGroup("<user id>", thread_id=thread_id)
+    thread.remove_participant("<user id>")
-
+    # Will add the users with IDs `<1st user id>`, `<2nd user id>` and `<3th user id>` to the group
-    # Will add the user with ID `<user id>` to the thread
+    thread.add_participants(["<1st user id>", "<2nd user id>", "<3rd user id>"])
-    client.addUsersToGroup("<user id>", thread_id=thread_id)
+    # Will change the title of the group to `<title>`
-
+    thread.set_title("<title>")
-    # Will add the users with IDs `<1st user id>`, `<2nd user id>` and `<3th user id>` to the thread
-    client.addUsersToGroup(
-        ["<1st user id>", "<2nd user id>", "<3rd user id>"], thread_id=thread_id
-    )
 
 
-# Will change the nickname of the user `<user_id>` to `<new nickname>`
+# Will change the nickname of the user `<user id>` to `<new nickname>`
-client.changeNickname(
+thread.set_nickname(fbchat.User(session=session, id="<user id>"), "<new nickname>")
-    "<new nickname>", "<user id>", thread_id=thread_id, thread_type=thread_type
-)
 
-# Will change the title of the thread to `<title>`
+# Will set the typing status of the thread
-client.changeThreadTitle("<title>", thread_id=thread_id, thread_type=thread_type)
+thread.start_typing()
 
-# Will set the typing status of the thread to `TYPING`
+# Will change the thread color to #0084ff
-client.setTypingStatus(
+thread.set_color("#0084ff")
-    TypingStatus.TYPING, thread_id=thread_id, thread_type=thread_type
-)
-
-# Will change the thread color to `MESSENGER_BLUE`
-client.changeThreadColor(ThreadColor.MESSENGER_BLUE, thread_id=thread_id)
 
 # Will change the thread emoji to `👍`
-client.changeThreadEmoji("👍", thread_id=thread_id)
+thread.set_emoji("👍")
+
+message = fbchat.Message(thread=thread, id="<message id>")
 
 # Will react to a message with a 😍 emoji
-client.reactToMessage("<message id>", MessageReaction.LOVE)
+message.react("😍")

examples/keepbot.py

@@ -1,13 +1,13 @@
-# -*- coding: UTF-8 -*-
+# This example uses the `blinker` library to dispatch events. See echobot.py for how
-
+# this could be done differenly. The decision is entirely up to you!
-from fbchat import log, Client
+import fbchat
-from fbchat.models import *
+import blinker
 
 # Change this to your group id
 old_thread_id = "1234567890"
 
 # Change these to match your liking
-old_color = ThreadColor.MESSENGER_BLUE
+old_color = "#0084ff"
 old_emoji = "👍"
 old_title = "Old group chat name"
 old_nicknames = {
@@ -17,67 +17,76 @@ old_nicknames = {
     "12345678904": "User nr. 4's nickname",
 }
 
+# Create a blinker signal
+events = blinker.Signal()
 
-class KeepBot(Client):
+# Register various event handlers on the signal
-    def onColorChange(self, author_id, new_color, thread_id, thread_type, **kwargs):
+@events.connect_via(fbchat.ColorSet)
-        if old_thread_id == thread_id and old_color != new_color:
+def on_color_set(sender, event: fbchat.ColorSet):
-            log.info(
+    if old_thread_id != event.thread.id:
-                "{} changed the thread color. It will be changed back".format(author_id)
+        return
+    if old_color != event.color:
+        print(f"{event.author.id} changed the thread color. It will be changed back")
+        event.thread.set_color(old_color)
+
+
+@events.connect_via(fbchat.EmojiSet)
+def on_emoji_set(sender, event: fbchat.EmojiSet):
+    if old_thread_id != event.thread.id:
+        return
+    if old_emoji != event.emoji:
+        print(f"{event.author.id} changed the thread emoji. It will be changed back")
+        event.thread.set_emoji(old_emoji)
+
+
+@events.connect_via(fbchat.TitleSet)
+def on_title_set(sender, event: fbchat.TitleSet):
+    if old_thread_id != event.thread.id:
+        return
+    if old_title != event.title:
+        print(f"{event.author.id} changed the thread title. It will be changed back")
+        event.thread.set_title(old_title)
+
+
+@events.connect_via(fbchat.NicknameSet)
+def on_nickname_set(sender, event: fbchat.NicknameSet):
+    if old_thread_id != event.thread.id:
+        return
+    old_nickname = old_nicknames.get(event.subject.id)
+    if old_nickname != event.nickname:
+        print(
+            f"{event.author.id} changed {event.subject.id}'s' nickname."
+            " It will be changed back"
         )
-            self.changeThreadColor(old_color, thread_id=thread_id)
+        event.thread.set_nickname(event.subject.id, old_nickname)
 
-    def onEmojiChange(self, author_id, new_emoji, thread_id, thread_type, **kwargs):
-        if old_thread_id == thread_id and new_emoji != old_emoji:
-            log.info(
-                "{} changed the thread emoji. It will be changed back".format(author_id)
-            )
-            self.changeThreadEmoji(old_emoji, thread_id=thread_id)
 
-    def onPeopleAdded(self, added_ids, author_id, thread_id, **kwargs):
+@events.connect_via(fbchat.PeopleAdded)
-        if old_thread_id == thread_id and author_id != self.uid:
+def on_people_added(sender, event: fbchat.PeopleAdded):
-            log.info("{} got added. They will be removed".format(added_ids))
+    if old_thread_id != event.thread.id:
-            for added_id in added_ids:
+        return
-                self.removeUserFromGroup(added_id, thread_id=thread_id)
+    if event.author.id != session.user.id:
+        print(f"{', '.join(x.id for x in event.added)} got added. They will be removed")
+        for added in event.added:
+            event.thread.remove_participant(added.id)
 
-    def onPersonRemoved(self, removed_id, author_id, thread_id, **kwargs):
+
+@events.connect_via(fbchat.PersonRemoved)
+def on_person_removed(sender, event: fbchat.PersonRemoved):
+    if old_thread_id != event.thread.id:
+        return
     # No point in trying to add ourself
-        if (
+    if event.removed.id == session.user.id:
-            old_thread_id == thread_id
+        return
-            and removed_id != self.uid
+    if event.author.id != session.user.id:
-            and author_id != self.uid
+        print(f"{event.removed.id} got removed. They will be re-added")
-        ):
+        event.thread.add_participants([event.removed.id])
-            log.info("{} got removed. They will be re-added".format(removed_id))
-            self.addUsersToGroup(removed_id, thread_id=thread_id)
-
-    def onTitleChange(self, author_id, new_title, thread_id, thread_type, **kwargs):
-        if old_thread_id == thread_id and old_title != new_title:
-            log.info(
-                "{} changed the thread title. It will be changed back".format(author_id)
-            )
-            self.changeThreadTitle(
-                old_title, thread_id=thread_id, thread_type=thread_type
-            )
-
-    def onNicknameChange(
-        self, author_id, changed_for, new_nickname, thread_id, thread_type, **kwargs
-    ):
-        if (
-            old_thread_id == thread_id
-            and changed_for in old_nicknames
-            and old_nicknames[changed_for] != new_nickname
-        ):
-            log.info(
-                "{} changed {}'s' nickname. It will be changed back".format(
-                    author_id, changed_for
-                )
-            )
-            self.changeNickname(
-                old_nicknames[changed_for],
-                changed_for,
-                thread_id=thread_id,
-                thread_type=thread_type,
-            )
 
 
-client = KeepBot("<email>", "<password>")
+# Login, and start listening for events
-client.listen()
+session = fbchat.Session.login("<email>", "<password>")
+listener = fbchat.Listener(session=session, chat_on=False, foreground=False)
+
+for event in listener.listen():
+    # Dispatch the event to the subscribed handlers
+    events.send(type(event), event=event)

examples/removebot.py

@@ -1,25 +1,17 @@
-# -*- coding: UTF-8 -*-
+import fbchat
-
-from fbchat import log, Client
-from fbchat.models import *
 
 
-class RemoveBot(Client):
+def on_message(event):
-    def onMessage(self, author_id, message_object, thread_id, thread_type, **kwargs):
     # We can only kick people from group chats, so no need to try if it's a user chat
-        if message_object.text == "Remove me!" and thread_type == ThreadType.GROUP:
+    if not isinstance(event.thread, fbchat.Group):
-            log.info("{} will be removed from {}".format(author_id, thread_id))
+        return
-            self.removeUserFromGroup(author_id, thread_id=thread_id)
+    if event.message.text == "Remove me!":
-        else:
+        print(f"{event.author.id} will be removed from {event.thread.id}")
-            # Sends the data to the inherited onMessage, so that we can still see when a message is recieved
+        event.thread.remove_participant(event.author.id)
-            super(RemoveBot, self).onMessage(
-                author_id=author_id,
-                message_object=message_object,
-                thread_id=thread_id,
-                thread_type=thread_type,
-                **kwargs
-            )
 
 
-client = RemoveBot("<email>", "<password>")
+session = fbchat.Session.login("<email>", "<password>")
-client.listen()
+listener = fbchat.Listener(session=session, chat_on=False, foreground=False)
+for event in listener.listen():
+    if isinstance(event, fbchat.MessageEvent):
+        on_message(event)

examples/session_handling.py

@@ -0,0 +1,42 @@
+# TODO: Consider adding Session.from_file and Session.to_file,
+# which would make this example a lot easier!
+
+import atexit
+import json
+import getpass
+import fbchat
+
+
+def load_cookies(filename):
+    try:
+        # Load cookies from file
+        with open(filename) as f:
+            return json.load(f)
+    except FileNotFoundError:
+        return  # No cookies yet
+
+
+def save_cookies(filename, cookies):
+    with open(filename, "w") as f:
+        json.dump(cookies, f)
+
+
+def load_session(cookies):
+    if not cookies:
+        return
+    try:
+        return fbchat.Session.from_cookies(cookies)
+    except fbchat.FacebookError:
+        return  # Failed loading from cookies
+
+
+cookies = load_cookies("session.json")
+session = load_session(cookies)
+if not session:
+    # Session could not be loaded, login instead!
+    session = fbchat.Session.login("<email>", getpass.getpass())
+
+# Save session cookies to file when the program exits
+atexit.register(lambda: save_cookies("session.json", session.get_cookies()))
+
+# Do stuff with session here

fbchat/__init__.py

@@ -1,25 +1,129 @@
-# -*- coding: UTF-8 -*-
+"""Facebook Messenger for Python.
-"""Facebook Chat (Messenger) for Python
 
-:copyright: (c) 2015 - 2019 by Taehoon Kim
+Copyright:
-:license: BSD 3-Clause, see LICENSE for more details.
+    (c) 2015 - 2018 by Taehoon Kim
+    (c) 2018 - 2020 by Mads Marquart
+
+License:
+    BSD 3-Clause, see LICENSE for more details.
 """
-from __future__ import unicode_literals
 
-# These imports are far too general, but they're needed for backwards compatbility.
+import logging as _logging
-from .utils import *
+
-from .graphql import *
+# Set default logging handler to avoid "No handler found" warnings.
-from .models import *
+_logging.getLogger(__name__).addHandler(_logging.NullHandler())
+
+# The order of these is somewhat significant, e.g. User has to be imported after Thread!
+from . import _common, _util
+from ._exception import (
+    FacebookError,
+    HTTPError,
+    ParseError,
+    ExternalError,
+    GraphQLError,
+    InvalidParameters,
+    NotLoggedIn,
+    PleaseRefresh,
+)
+from ._session import Session
+from ._threads import (
+    ThreadABC,
+    Thread,
+    User,
+    UserData,
+    Group,
+    GroupData,
+    Page,
+    PageData,
+)
+
+# Models
+from ._models import (
+    Image,
+    ThreadLocation,
+    ActiveStatus,
+    Attachment,
+    UnsentMessage,
+    ShareAttachment,
+    LocationAttachment,
+    LiveLocationAttachment,
+    Sticker,
+    FileAttachment,
+    AudioAttachment,
+    ImageAttachment,
+    VideoAttachment,
+    Poll,
+    PollOption,
+    GuestStatus,
+    Plan,
+    PlanData,
+    QuickReply,
+    QuickReplyText,
+    QuickReplyLocation,
+    QuickReplyPhoneNumber,
+    QuickReplyEmail,
+    EmojiSize,
+    Mention,
+    Message,
+    MessageSnippet,
+    MessageData,
+)
+
+# Events
+from ._events import (
+    # _common
+    Event,
+    UnknownEvent,
+    ThreadEvent,
+    Connect,
+    Disconnect,
+    # _client_payload
+    ReactionEvent,
+    UserStatusEvent,
+    LiveLocationEvent,
+    UnsendEvent,
+    MessageReplyEvent,
+    # _delta_class
+    PeopleAdded,
+    PersonRemoved,
+    TitleSet,
+    UnfetchedThreadEvent,
+    MessagesDelivered,
+    ThreadsRead,
+    MessageEvent,
+    ThreadFolder,
+    # _delta_type
+    ColorSet,
+    EmojiSet,
+    NicknameSet,
+    AdminsAdded,
+    AdminsRemoved,
+    ApprovalModeSet,
+    CallStarted,
+    CallEnded,
+    CallJoined,
+    PollCreated,
+    PollVoted,
+    PlanCreated,
+    PlanEnded,
+    PlanEdited,
+    PlanDeleted,
+    PlanResponded,
+    # __init__
+    Typing,
+    FriendRequest,
+    Presence,
+)
+from ._listen import Listener
+
 from ._client import Client
 
-__title__ = "fbchat"
+__version__ = "2.0.0a4"
-__version__ = "1.6.5"
-__description__ = "Facebook Chat (Messenger) for Python"
 
-__copyright__ = "Copyright 2015 - 2019 by Taehoon Kim"
+__all__ = ("Session", "Listener", "Client")
-__license__ = "BSD 3-Clause"
 
-__author__ = "Taehoon Kim; Moreels Pieter-Jan; Mads Marquart"
-__email__ = "carpedm20@gmail.com"
 
-__all__ = ["Client"]
+from . import _fix_module_metadata
+
+_fix_module_metadata.fixup_module_metadata(globals())
+del _fix_module_metadata

fbchat/_attachment.py

@@ -1,48 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-
-
-@attr.s(cmp=False)
-class Attachment(object):
-    """Represents a Facebook attachment"""
-
-    #: The attachment ID
-    uid = attr.ib(None)
-
-
-@attr.s(cmp=False)
-class UnsentMessage(Attachment):
-    """Represents an unsent message attachment"""
-
-
-@attr.s(cmp=False)
-class ShareAttachment(Attachment):
-    """Represents a shared item (eg. URL) that has been sent as a Facebook attachment"""
-
-    #: ID of the author of the shared post
-    author = attr.ib(None)
-    #: Target URL
-    url = attr.ib(None)
-    #: Original URL if Facebook redirects the URL
-    original_url = attr.ib(None)
-    #: Title of the attachment
-    title = attr.ib(None)
-    #: Description of the attachment
-    description = attr.ib(None)
-    #: Name of the source
-    source = attr.ib(None)
-    #: URL of the attachment image
-    image_url = attr.ib(None)
-    #: URL of the original image if Facebook uses `safe_image`
-    original_image_url = attr.ib(None)
-    #: Width of the image
-    image_width = attr.ib(None)
-    #: Height of the image
-    image_height = attr.ib(None)
-    #: List of additional attachments
-    attachments = attr.ib(factory=list, converter=lambda x: [] if x is None else x)
-
-    # Put here for backwards compatibility, so that the init argument order is preserved
-    uid = attr.ib(None)

fbchat/_common.py

@@ -0,0 +1,11 @@
+import sys
+import attr
+import logging
+
+log = logging.getLogger("fbchat")
+
+# Enable kw_only if the python version supports it
+kw_only = sys.version_info[:2] > (3, 5)
+
+#: Default attrs settings for classes
+attrs_default = attr.s(frozen=True, slots=True, kw_only=kw_only)

fbchat/_core.py

@@ -1,12 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import aenum
-
-
-class Enum(aenum.Enum):
-    """Used internally by fbchat to support enumerations"""
-
-    def __repr__(self):
-        # For documentation:
-        return "{}.{}".format(type(self).__name__, self.name)

fbchat/_events/__init__.py

@@ -0,0 +1,132 @@
+import attr
+import datetime
+from ._common import attrs_event, Event, UnknownEvent, ThreadEvent
+from ._client_payload import *
+from ._delta_class import *
+from ._delta_type import *
+
+from .. import _exception, _threads, _models
+
+from typing import Mapping
+
+
+@attrs_event
+class Typing(ThreadEvent):
+    """Somebody started/stopped typing in a thread."""
+
+    #: ``True`` if the user started typing, ``False`` if they stopped
+    status = attr.ib(type=bool)
+
+    @classmethod
+    def _parse_orca(cls, session, data):
+        author = _threads.User(session=session, id=str(data["sender_fbid"]))
+        status = data["state"] == 1
+        return cls(author=author, thread=author, status=status)
+
+    @classmethod
+    def _parse_thread_typing(cls, session, data):
+        author = _threads.User(session=session, id=str(data["sender_fbid"]))
+        thread = _threads.Group(session=session, id=str(data["thread"]))
+        status = data["state"] == 1
+        return cls(author=author, thread=thread, status=status)
+
+
+@attrs_event
+class FriendRequest(Event):
+    """Somebody sent a friend request."""
+
+    #: The user that sent the request
+    author = attr.ib(type="_threads.User")
+
+    @classmethod
+    def _parse(cls, session, data):
+        author = _threads.User(session=session, id=str(data["from"]))
+        return cls(author=author)
+
+
+@attrs_event
+class Presence(Event):
+    """The list of active statuses was updated.
+
+    Chat online presence update.
+    """
+
+    # TODO: Document this better!
+
+    #: User ids mapped to their active status
+    statuses = attr.ib(type=Mapping[str, "_models.ActiveStatus"])
+    #: ``True`` if the list is fully updated and ``False`` if it's partially updated
+    full = attr.ib(type=bool)
+
+    @classmethod
+    def _parse(cls, session, data):
+        statuses = {
+            str(d["u"]): _models.ActiveStatus._from_orca_presence(d)
+            for d in data["list"]
+        }
+        return cls(statuses=statuses, full=data["list_type"] == "full")
+
+
+@attrs_event
+class Connect(Event):
+    """The client was connected to Facebook.
+
+    This is not guaranteed to be triggered the same amount of times `Disconnect`!
+    """
+
+
+@attrs_event
+class Disconnect(Event):
+    """The client lost the connection to Facebook.
+
+    This is not guaranteed to be triggered the same amount of times `Connect`!
+    """
+
+    #: The reason / error string for the disconnect
+    reason = attr.ib(type=str)
+
+
+def parse_events(session, topic, data):
+    # See Mqtt._configure_connect_options for information about these topics
+    try:
+        if topic == "/t_ms":
+            # `deltas` will always be available, since we're filtering out the things
+            # that don't have it earlier in the MQTT listener
+            for delta in data["deltas"]:
+                if delta["class"] == "ClientPayload":
+                    yield from parse_client_payloads(session, delta)
+                    continue
+                try:
+                    event = parse_delta(session, delta)
+                    if event:  # Skip `None`
+                        yield event
+                except _exception.ParseError:
+                    raise
+                except Exception as e:
+                    raise _exception.ParseError(
+                        "Error parsing delta", data=delta
+                    ) from e
+
+        elif topic == "/thread_typing":
+            yield Typing._parse_thread_typing(session, data)
+
+        elif topic == "/orca_typing_notifications":
+            yield Typing._parse_orca(session, data)
+
+        elif topic == "/legacy_web":
+            if data["type"] == "jewel_requests_add":
+                yield FriendRequest._parse(session, data)
+            else:
+                yield UnknownEvent(source="/legacy_web", data=data)
+
+        elif topic == "/orca_presence":
+            yield Presence._parse(session, data)
+
+        else:
+            yield UnknownEvent(source=topic, data=data)
+    except _exception.ParseError:
+        raise
+    except Exception as e:
+        raise _exception.ParseError(
+            "Error parsing MQTT topic {}".format(topic), data=data
+        ) from e

fbchat/_events/_client_payload.py

@@ -0,0 +1,136 @@
+import attr
+import datetime
+from ._common import attrs_event, UnknownEvent, ThreadEvent
+from .. import _exception, _util, _threads, _models
+
+from typing import Optional
+
+
+@attrs_event
+class ReactionEvent(ThreadEvent):
+    """Somebody reacted to a message."""
+
+    #: Message that the user reacted to
+    message = attr.ib(type="_models.Message")
+
+    reaction = attr.ib(type=Optional[str])
+    """The reaction.
+
+    Not limited to the ones in `Message.react`.
+
+    If ``None``, the reaction was removed.
+    """
+
+    @classmethod
+    def _parse(cls, session, data):
+        thread = cls._get_thread(session, data)
+        return cls(
+            author=_threads.User(session=session, id=str(data["userId"])),
+            thread=thread,
+            message=_models.Message(thread=thread, id=data["messageId"]),
+            reaction=data["reaction"] if data["action"] == 0 else None,
+        )
+
+
+@attrs_event
+class UserStatusEvent(ThreadEvent):
+    #: Whether the user was blocked or unblocked
+    blocked = attr.ib(type=bool)
+
+    @classmethod
+    def _parse(cls, session, data):
+        return cls(
+            author=_threads.User(session=session, id=str(data["actorFbid"])),
+            thread=cls._get_thread(session, data),
+            blocked=not data["canViewerReply"],
+        )
+
+
+@attrs_event
+class LiveLocationEvent(ThreadEvent):
+    """Somebody sent live location info."""
+
+    # TODO: This!
+
+    @classmethod
+    def _parse(cls, session, data):
+        from . import _location
+
+        thread = cls._get_thread(session, data)
+        for location_data in data["messageLiveLocations"]:
+            message = _models.Message(thread=thread, id=data["messageId"])
+            author = _threads.User(session=session, id=str(location_data["senderId"]))
+            location = _location.LiveLocationAttachment._from_pull(location_data)
+
+        return None
+
+
+@attrs_event
+class UnsendEvent(ThreadEvent):
+    """Somebody unsent a message (which deletes it for everyone)."""
+
+    #: The unsent message
+    message = attr.ib(type="_models.Message")
+    #: When the message was unsent
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        thread = cls._get_thread(session, data)
+        return cls(
+            author=_threads.User(session=session, id=str(data["senderID"])),
+            thread=thread,
+            message=_models.Message(thread=thread, id=data["messageID"]),
+            at=_util.millis_to_datetime(data["deletionTimestamp"]),
+        )
+
+
+@attrs_event
+class MessageReplyEvent(ThreadEvent):
+    """Somebody replied to a message."""
+
+    #: The sent message
+    message = attr.ib(type="_models.MessageData")
+    #: The message that was replied to
+    replied_to = attr.ib(type="_models.MessageData")
+
+    @classmethod
+    def _parse(cls, session, data):
+        metadata = data["message"]["messageMetadata"]
+        thread = cls._get_thread(session, metadata)
+        return cls(
+            author=_threads.User(session=session, id=str(metadata["actorFbId"])),
+            thread=thread,
+            message=_models.MessageData._from_reply(thread, data["message"]),
+            replied_to=_models.MessageData._from_reply(
+                thread, data["repliedToMessage"]
+            ),
+        )
+
+
+def parse_client_delta(session, data):
+    if "deltaMessageReaction" in data:
+        return ReactionEvent._parse(session, data["deltaMessageReaction"])
+    elif "deltaChangeViewerStatus" in data:
+        # TODO: Parse all `reason`
+        if data["deltaChangeViewerStatus"]["reason"] == 2:
+            return UserStatusEvent._parse(session, data["deltaChangeViewerStatus"])
+    elif "liveLocationData" in data:
+        return LiveLocationEvent._parse(session, data["liveLocationData"])
+    elif "deltaRecallMessageData" in data:
+        return UnsendEvent._parse(session, data["deltaRecallMessageData"])
+    elif "deltaMessageReply" in data:
+        return MessageReplyEvent._parse(session, data["deltaMessageReply"])
+    return UnknownEvent(source="client payload", data=data)
+
+
+def parse_client_payloads(session, data):
+    payload = _util.parse_json("".join(chr(z) for z in data["payload"]))
+
+    try:
+        for delta in payload["deltas"]:
+            yield parse_client_delta(session, delta)
+    except _exception.ParseError:
+        raise
+    except Exception as e:
+        raise _exception.ParseError("Error parsing ClientPayload", data=payload) from e

fbchat/_events/_common.py

@@ -0,0 +1,62 @@
+import attr
+from .._common import kw_only
+from .. import _exception, _util, _threads
+
+from typing import Any
+
+#: Default attrs settings for events
+attrs_event = attr.s(slots=True, kw_only=kw_only, frozen=True)
+
+
+@attrs_event
+class Event:
+    """Base class for all events."""
+
+    @staticmethod
+    def _get_thread(session, data):
+        # TODO: Handle pages? Is it even possible?
+        key = data["threadKey"]
+
+        if "threadFbId" in key:
+            return _threads.Group(session=session, id=str(key["threadFbId"]))
+        elif "otherUserFbId" in key:
+            return _threads.User(session=session, id=str(key["otherUserFbId"]))
+        raise _exception.ParseError("Could not find thread data", data=data)
+
+
+@attrs_event
+class UnknownEvent(Event):
+    """Represent an unknown event."""
+
+    #: Some data describing the unknown event's origin
+    source = attr.ib(type=str)
+    #: The unknown data. This cannot be relied on, it's only for debugging purposes.
+    data = attr.ib(type=Any)
+
+    @classmethod
+    def _parse(cls, session, data):
+        raise NotImplementedError
+
+
+@attrs_event
+class ThreadEvent(Event):
+    """Represent an event that was done by a user/page in a thread."""
+
+    #: The person who did the action
+    author = attr.ib(type="_threads.User")  # Or Union[User, Page]?
+    #: Thread that the action was done in
+    thread = attr.ib(type="_threads.ThreadABC")
+
+    @classmethod
+    def _parse_metadata(cls, session, data):
+        metadata = data["messageMetadata"]
+        author = _threads.User(session=session, id=metadata["actorFbId"])
+        thread = cls._get_thread(session, metadata)
+        at = _util.millis_to_datetime(int(metadata["timestamp"]))
+        return author, thread, at
+
+    @classmethod
+    def _parse_fetch(cls, session, data):
+        author = _threads.User(session=session, id=data["message_sender"]["id"])
+        at = _util.millis_to_datetime(int(data["timestamp_precise"]))
+        return author, at

fbchat/_events/_delta_class.py

@@ -0,0 +1,214 @@
+import attr
+import datetime
+from ._common import attrs_event, Event, UnknownEvent, ThreadEvent
+from . import _delta_type
+from .. import _util, _threads, _models
+
+from typing import Sequence, Optional
+
+
+@attrs_event
+class PeopleAdded(ThreadEvent):
+    """somebody added people to a group thread."""
+
+    # TODO: Add message id
+
+    thread = attr.ib(type="_threads.Group")  # Set the correct type
+    #: The people who got added
+    added = attr.ib(type=Sequence["_threads.User"])
+    #: When the people were added
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        added = [
+            # TODO: Parse user name
+            _threads.User(session=session, id=x["userFbId"])
+            for x in data["addedParticipants"]
+        ]
+        return cls(author=author, thread=thread, added=added, at=at)
+
+
+@attrs_event
+class PersonRemoved(ThreadEvent):
+    """Somebody removed a person from a group thread."""
+
+    # TODO: Add message id
+
+    thread = attr.ib(type="_threads.Group")  # Set the correct type
+    #: Person who got removed
+    removed = attr.ib(type="_models.Message")
+    #: When the person were removed
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        removed = _threads.User(session=session, id=data["leftParticipantFbId"])
+        return cls(author=author, thread=thread, removed=removed, at=at)
+
+
+@attrs_event
+class TitleSet(ThreadEvent):
+    """Somebody changed a group's title."""
+
+    thread = attr.ib(type="_threads.Group")  # Set the correct type
+    #: The new title. If ``None``, the title was removed
+    title = attr.ib(type=Optional[str])
+    #: When the title was set
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        return cls(author=author, thread=thread, title=data["name"] or None, at=at)
+
+
+@attrs_event
+class UnfetchedThreadEvent(Event):
+    """A message was received, but the data must be fetched manually.
+
+    Use `Message.fetch` to retrieve the message data.
+
+    This is usually used when somebody changes the group's photo, or when a new pending
+    group is created.
+    """
+
+    # TODO: Present this in a way that users can fetch the changed group photo easily
+
+    #: The thread the message was sent to
+    thread = attr.ib(type="_threads.ThreadABC")
+    #: The message
+    message = attr.ib(type=Optional["_models.Message"])
+
+    @classmethod
+    def _parse(cls, session, data):
+        thread = cls._get_thread(session, data)
+        message = None
+        if "messageId" in data:
+            message = _models.Message(thread=thread, id=data["messageId"])
+        return cls(thread=thread, message=message)
+
+
+@attrs_event
+class MessagesDelivered(ThreadEvent):
+    """Somebody marked messages as delivered in a thread."""
+
+    #: The messages that were marked as delivered
+    messages = attr.ib(type=Sequence["_models.Message"])
+    #: When the messages were delivered
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        thread = cls._get_thread(session, data)
+        if "actorFbId" in data:
+            author = _threads.User(session=session, id=data["actorFbId"])
+        else:
+            author = thread
+        messages = [_models.Message(thread=thread, id=x) for x in data["messageIds"]]
+        at = _util.millis_to_datetime(int(data["deliveredWatermarkTimestampMs"]))
+        return cls(author=author, thread=thread, messages=messages, at=at)
+
+
+@attrs_event
+class ThreadsRead(Event):
+    """Somebody marked threads as read/seen."""
+
+    #: The person who marked the threads as read
+    author = attr.ib(type="_threads.ThreadABC")
+    #: The threads that were marked as read
+    threads = attr.ib(type=Sequence["_threads.ThreadABC"])
+    #: When the threads were read
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse_read_receipt(cls, session, data):
+        author = _threads.User(session=session, id=data["actorFbId"])
+        thread = cls._get_thread(session, data)
+        at = _util.millis_to_datetime(int(data["actionTimestampMs"]))
+        return cls(author=author, threads=[thread], at=at)
+
+    @classmethod
+    def _parse(cls, session, data):
+        threads = [
+            cls._get_thread(session, {"threadKey": x}) for x in data["threadKeys"]
+        ]
+        at = _util.millis_to_datetime(int(data["actionTimestamp"]))
+        return cls(author=session.user, threads=threads, at=at)
+
+
+@attrs_event
+class MessageEvent(ThreadEvent):
+    """Somebody sent a message to a thread."""
+
+    #: The sent message
+    message = attr.ib(type="_models.Message")
+    #: When the threads were read
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        message = _models.MessageData._from_pull(
+            thread, data, author=author.id, created_at=at,
+        )
+        return cls(author=author, thread=thread, message=message, at=at)
+
+
+@attrs_event
+class ThreadFolder(Event):
+    """A thread was created in a folder.
+
+    Somebody that isn't connected with you on either Facebook or Messenger sends a
+    message. After that, you need to use `ThreadABC.fetch_messages` to actually read it.
+    """
+
+    # TODO: Finish this
+
+    #: The created thread
+    thread = attr.ib(type="_threads.ThreadABC")
+    #: The folder/location
+    folder = attr.ib(type="_models.ThreadLocation")
+
+    @classmethod
+    def _parse(cls, session, data):
+        thread = cls._get_thread(session, data)
+        folder = _models.ThreadLocation._parse(data["folder"])
+        return cls(thread=thread, folder=folder)
+
+
+def parse_delta(session, data):
+    class_ = data["class"]
+    if class_ == "AdminTextMessage":
+        return _delta_type.parse_admin_message(session, data)
+    elif class_ == "ParticipantsAddedToGroupThread":
+        return PeopleAdded._parse(session, data)
+    elif class_ == "ParticipantLeftGroupThread":
+        return PersonRemoved._parse(session, data)
+    elif class_ == "MarkFolderSeen":
+        # TODO: Finish this
+        folders = [_models.ThreadLocation._parse(folder) for folder in data["folders"]]
+        at = _util.millis_to_datetime(int(data["timestamp"]))
+        return None
+    elif class_ == "ThreadName":
+        return TitleSet._parse(session, data)
+    elif class_ == "ForcedFetch":
+        return UnfetchedThreadEvent._parse(session, data)
+    elif class_ == "DeliveryReceipt":
+        return MessagesDelivered._parse(session, data)
+    elif class_ == "ReadReceipt":
+        return ThreadsRead._parse_read_receipt(session, data)
+    elif class_ == "MarkRead":
+        return ThreadsRead._parse(session, data)
+    elif class_ == "NoOp":
+        # Skip "no operation" events
+        return None
+    elif class_ == "NewMessage":
+        return MessageEvent._parse(session, data)
+    elif class_ == "ThreadFolder":
+        return ThreadFolder._parse(session, data)
+    elif class_ == "ClientPayload":
+        raise ValueError("This is implemented in `parse_events`")
+    return UnknownEvent(source="Delta class", data=data)

fbchat/_events/_delta_type.py

@@ -0,0 +1,331 @@
+import attr
+import datetime
+from ._common import attrs_event, Event, UnknownEvent, ThreadEvent
+from .. import _util, _threads, _models
+
+from typing import Sequence, Optional
+
+
+@attrs_event
+class ColorSet(ThreadEvent):
+    """Somebody set the color in a thread."""
+
+    #: The new color. Not limited to the ones in `ThreadABC.set_color`
+    color = attr.ib(type=str)
+    #: When the color was set
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        color = _threads.ThreadABC._parse_color(data["untypedData"]["theme_color"])
+        return cls(author=author, thread=thread, color=color, at=at)
+
+
+@attrs_event
+class EmojiSet(ThreadEvent):
+    """Somebody set the emoji in a thread."""
+
+    #: The new emoji
+    emoji = attr.ib(type=str)
+    #: When the emoji was set
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        emoji = data["untypedData"]["thread_icon"]
+        return cls(author=author, thread=thread, emoji=emoji, at=at)
+
+
+@attrs_event
+class NicknameSet(ThreadEvent):
+    """Somebody set the nickname of a person in a thread."""
+
+    #: The person whose nickname was set
+    subject = attr.ib(type=str)
+    #: The new nickname. If ``None``, the nickname was cleared
+    nickname = attr.ib(type=Optional[str])
+    #: When the nickname was set
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        subject = _threads.User(
+            session=session, id=data["untypedData"]["participant_id"]
+        )
+        nickname = data["untypedData"]["nickname"] or None  # None if ""
+        return cls(
+            author=author, thread=thread, subject=subject, nickname=nickname, at=at
+        )
+
+
+@attrs_event
+class AdminsAdded(ThreadEvent):
+    """Somebody added admins to a group."""
+
+    #: The people that were set as admins
+    added = attr.ib(type=Sequence["_threads.User"])
+    #: When the admins were added
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        subject = _threads.User(session=session, id=data["untypedData"]["TARGET_ID"])
+        return cls(author=author, thread=thread, added=[subject], at=at)
+
+
+@attrs_event
+class AdminsRemoved(ThreadEvent):
+    """Somebody removed admins from a group."""
+
+    #: The people that were removed as admins
+    removed = attr.ib(type=Sequence["_threads.User"])
+    #: When the admins were removed
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        subject = _threads.User(session=session, id=data["untypedData"]["TARGET_ID"])
+        return cls(author=author, thread=thread, removed=[subject], at=at)
+
+
+@attrs_event
+class ApprovalModeSet(ThreadEvent):
+    """Somebody changed the approval mode in a group."""
+
+    require_admin_approval = attr.ib(type=bool)
+    #: When the approval mode was set
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        raa = data["untypedData"]["APPROVAL_MODE"] == "1"
+        return cls(author=author, thread=thread, require_admin_approval=raa, at=at)
+
+
+@attrs_event
+class CallStarted(ThreadEvent):
+    """Somebody started a call."""
+
+    #: When the call was started
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        return cls(author=author, thread=thread, at=at)
+
+
+@attrs_event
+class CallEnded(ThreadEvent):
+    """Somebody ended a call."""
+
+    #: How long the call took
+    duration = attr.ib(type=datetime.timedelta)
+    #: When the call ended
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        duration = _util.seconds_to_timedelta(int(data["untypedData"]["call_duration"]))
+        return cls(author=author, thread=thread, duration=duration, at=at)
+
+
+@attrs_event
+class CallJoined(ThreadEvent):
+    """Somebody joined a call."""
+
+    #: When the call ended
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        return cls(author=author, thread=thread, at=at)
+
+
+@attrs_event
+class PollCreated(ThreadEvent):
+    """Somebody created a group poll."""
+
+    #: The new poll
+    poll = attr.ib(type="_models.Poll")
+    #: When the poll was created
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        poll_data = _util.parse_json(data["untypedData"]["question_json"])
+        poll = _models.Poll._from_graphql(session, poll_data)
+        return cls(author=author, thread=thread, poll=poll, at=at)
+
+
+@attrs_event
+class PollVoted(ThreadEvent):
+    """Somebody voted in a group poll."""
+
+    #: The updated poll
+    poll = attr.ib(type="_models.Poll")
+    #: Ids of the voted options
+    added_ids = attr.ib(type=Sequence[str])
+    #: Ids of the un-voted options
+    removed_ids = attr.ib(type=Sequence[str])
+    #: When the poll was voted in
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        poll_data = _util.parse_json(data["untypedData"]["question_json"])
+        poll = _models.Poll._from_graphql(session, poll_data)
+        added_ids = _util.parse_json(data["untypedData"]["added_option_ids"])
+        removed_ids = _util.parse_json(data["untypedData"]["removed_option_ids"])
+        return cls(
+            author=author,
+            thread=thread,
+            poll=poll,
+            added_ids=[str(x) for x in added_ids],
+            removed_ids=[str(x) for x in removed_ids],
+            at=at,
+        )
+
+
+@attrs_event
+class PlanCreated(ThreadEvent):
+    """Somebody created a plan in a group."""
+
+    #: The new plan
+    plan = attr.ib(type="_models.PlanData")
+    #: When the plan was created
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        plan = _models.PlanData._from_pull(session, data["untypedData"])
+        return cls(author=author, thread=thread, plan=plan, at=at)
+
+
+@attrs_event
+class PlanEnded(ThreadEvent):
+    """A plan ended."""
+
+    #: The ended plan
+    plan = attr.ib(type="_models.PlanData")
+    #: When the plan ended
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        plan = _models.PlanData._from_pull(session, data["untypedData"])
+        return cls(author=author, thread=thread, plan=plan, at=at)
+
+
+@attrs_event
+class PlanEdited(ThreadEvent):
+    """Somebody changed a plan in a group."""
+
+    #: The updated plan
+    plan = attr.ib(type="_models.PlanData")
+    #: When the plan was updated
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        plan = _models.PlanData._from_pull(session, data["untypedData"])
+        return cls(author=author, thread=thread, plan=plan, at=at)
+
+
+@attrs_event
+class PlanDeleted(ThreadEvent):
+    """Somebody removed a plan in a group."""
+
+    #: The removed plan
+    plan = attr.ib(type="_models.PlanData")
+    #: When the plan was removed
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        plan = _models.PlanData._from_pull(session, data["untypedData"])
+        return cls(author=author, thread=thread, plan=plan, at=at)
+
+
+@attrs_event
+class PlanResponded(ThreadEvent):
+    """Somebody responded to a plan in a group."""
+
+    #: The plan that was responded to
+    plan = attr.ib(type="_models.PlanData")
+    #: Whether the author will go to the plan or not
+    take_part = attr.ib(type=bool)
+    #: When the plan was removed
+    at = attr.ib(type=datetime.datetime)
+
+    @classmethod
+    def _parse(cls, session, data):
+        author, thread, at = cls._parse_metadata(session, data)
+        plan = _models.PlanData._from_pull(session, data["untypedData"])
+        take_part = data["untypedData"]["guest_status"] == "GOING"
+        return cls(author=author, thread=thread, plan=plan, take_part=take_part, at=at)
+
+
+def parse_admin_message(session, data):
+    type_ = data["type"]
+    if type_ == "change_thread_theme":
+        return ColorSet._parse(session, data)
+    elif type_ == "change_thread_icon":
+        return EmojiSet._parse(session, data)
+    elif type_ == "change_thread_nickname":
+        return NicknameSet._parse(session, data)
+    elif type_ == "change_thread_admins":
+        event_type = data["untypedData"]["ADMIN_EVENT"]
+        if event_type == "add_admin":
+            return AdminsAdded._parse(session, data)
+        elif event_type == "remove_admin":
+            return AdminsRemoved._parse(session, data)
+        else:
+            pass
+    elif type_ == "change_thread_approval_mode":
+        return ApprovalModeSet._parse(session, data)
+    elif type_ == "instant_game_update":
+        pass  # TODO: This
+    elif type_ == "messenger_call_log":  # Previously "rtc_call_log"
+        event_type = data["untypedData"]["event"]
+        if event_type == "group_call_started":
+            return CallStarted._parse(session, data)
+        elif event_type in ["group_call_ended", "one_on_one_call_ended"]:
+            return CallEnded._parse(session, data)
+        else:
+            pass
+    elif type_ == "participant_joined_group_call":
+        return CallJoined._parse(session, data)
+    elif type_ == "group_poll":
+        event_type = data["untypedData"]["event_type"]
+        if event_type == "question_creation":
+            return PollCreated._parse(session, data)
+        elif event_type == "update_vote":
+            return PollVoted._parse(session, data)
+        else:
+            pass
+    elif type_ == "lightweight_event_create":
+        return PlanCreated._parse(session, data)
+    elif type_ == "lightweight_event_notify":
+        return PlanEnded._parse(session, data)
+    elif type_ == "lightweight_event_update":
+        return PlanEdited._parse(session, data)
+    elif type_ == "lightweight_event_delete":
+        return PlanDeleted._parse(session, data)
+    elif type_ == "lightweight_event_rsvp":
+        return PlanResponded._parse(session, data)
+    return UnknownEvent(source="Delta type", data=data)

fbchat/_exception.py

@@ -1,32 +1,165 @@
-# -*- coding: UTF-8 -*-
+import attr
-from __future__ import unicode_literals
+import requests
+
+from typing import Any, Optional
+
+# Not frozen, since that doesn't work in PyPy
+@attr.s(slots=True, auto_exc=True)
+class FacebookError(Exception):
+    """Base class for all custom exceptions raised by ``fbchat``.
+
+    All exceptions in the module inherit this.
+    """
+
+    #: A message describing the error
+    message = attr.ib(type=str)
 
 
-class FBchatException(Exception):
+@attr.s(slots=True, auto_exc=True)
-    """Custom exception thrown by fbchat. All exceptions in the fbchat module inherits this"""
+class HTTPError(FacebookError):
+    """Base class for errors with the HTTP(s) connection to Facebook."""
+
+    #: The returned HTTP status code, if relevant
+    status_code = attr.ib(None, type=Optional[int])
+
+    def __str__(self):
+        if not self.status_code:
+            return self.message
+        return "Got {} response: {}".format(self.status_code, self.message)
 
 
-class FBchatFacebookError(FBchatException):
+@attr.s(slots=True, auto_exc=True)
+class ParseError(FacebookError):
+    """Raised when we fail parsing a response from Facebook.
+
+    This may contain sensitive data, so should not be logged to file.
+    """
+
+    data = attr.ib(type=Any)
+    """The data that triggered the error.
+
+    The format of this cannot be relied on, it's only for debugging purposes.
+    """
+
+    def __str__(self):
+        msg = "{}. Please report this, along with the data below!\n{}"
+        return msg.format(self.message, self.data)
+
+
+@attr.s(slots=True, auto_exc=True)
+class NotLoggedIn(FacebookError):
+    """Raised by Facebook if the client has been logged out."""
+
+
+@attr.s(slots=True, auto_exc=True)
+class ExternalError(FacebookError):
+    """Base class for errors that Facebook return."""
+
+    #: The error message that Facebook returned (Possibly in the user's own language)
+    description = attr.ib(type=str)
     #: The error code that Facebook returned
-    fb_error_code = None
+    code = attr.ib(None, type=Optional[int])
-    #: The error message that Facebook returned (In the user's own language)
-    fb_error_message = None
-    #: The status code that was sent in the http response (eg. 404) (Usually only set if not successful, aka. not 200)
-    request_status_code = None
 
-    def __init__(
+    def __str__(self):
-        self,
+        if self.code:
-        message,
+            return "#{} {}: {}".format(self.code, self.message, self.description)
-        fb_error_code=None,
+        return "{}: {}".format(self.message, self.description)
-        fb_error_message=None,
-        request_status_code=None,
-    ):
-        super(FBchatFacebookError, self).__init__(message)
-        """Thrown by fbchat when Facebook returns an error"""
-        self.fb_error_code = str(fb_error_code)
-        self.fb_error_message = fb_error_message
-        self.request_status_code = request_status_code
 
 
-class FBchatUserError(FBchatException):
+@attr.s(slots=True, auto_exc=True)
-    """Thrown by fbchat when wrong values are entered"""
+class GraphQLError(ExternalError):
+    """Raised by Facebook if there was an error in the GraphQL query."""
+
+    # TODO: Handle multiple errors
+
+    #: Query debug information
+    debug_info = attr.ib(None, type=Optional[str])
+
+    def __str__(self):
+        if self.debug_info:
+            return "{}, {}".format(super().__str__(), self.debug_info)
+        return super().__str__()
+
+
+@attr.s(slots=True, auto_exc=True)
+class InvalidParameters(ExternalError):
+    """Raised by Facebook if:
+
+    - Some function supplied invalid parameters.
+    - Some content is not found.
+    - Some content is no longer available.
+    """
+
+
+@attr.s(slots=True, auto_exc=True)
+class PleaseRefresh(ExternalError):
+    """Raised by Facebook if the client has been inactive for too long.
+
+    This error usually happens after 1-2 days of inactivity.
+    """
+
+    code = attr.ib(1357004)
+
+
+def handle_payload_error(j):
+    if "error" not in j:
+        return
+    code = j["error"]
+    if code == 1357001:
+        raise NotLoggedIn(j["errorSummary"])
+    elif code == 1357004:
+        error_cls = PleaseRefresh
+    elif code in (1357031, 1545010, 1545003):
+        error_cls = InvalidParameters
+    else:
+        error_cls = ExternalError
+    raise error_cls(j["errorSummary"], description=j["errorDescription"], code=code)
+
+
+def handle_graphql_errors(j):
+    errors = []
+    if j.get("error"):
+        errors = [j["error"]]
+    if "errors" in j:
+        errors = j["errors"]
+    if errors:
+        error = errors[0]  # TODO: Handle multiple errors
+        # TODO: Use `severity`
+        raise GraphQLError(
+            # TODO: What data is always available?
+            message=error.get("summary", "Unknown error"),
+            description=error.get("message") or error.get("description") or "",
+            code=error.get("code"),
+            debug_info=error.get("debug_info"),
+        )
+
+
+def handle_http_error(code):
+    if code == 404:
+        raise HTTPError(
+            "This might be because you provided an invalid id"
+            + " (Facebook usually require integer ids)",
+            status_code=code,
+        )
+    if code == 500:
+        raise HTTPError(
+            "There is probably an error on the endpoint, or it might be rate limited",
+            status_code=code,
+        )
+    if 400 <= code < 600:
+        raise HTTPError("Failed sending request", status_code=code)
+
+
+def handle_requests_error(e):
+    if isinstance(e, requests.ConnectionError):
+        raise HTTPError("Connection error") from e
+    if isinstance(e, requests.HTTPError):
+        pass  # Raised when using .raise_for_status, so should never happen
+    if isinstance(e, requests.URLRequired):
+        pass  # Should never happen, we always prove valid URLs
+    if isinstance(e, requests.TooManyRedirects):
+        pass  # TODO: Consider using allow_redirects=False to prevent this
+    if isinstance(e, requests.Timeout):
+        pass  # Should never happen, we don't set timeouts
+
+    raise HTTPError("Requests error") from e

fbchat/_file.py

@@ -1,197 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-from ._attachment import Attachment
-
-
-@attr.s(cmp=False)
-class FileAttachment(Attachment):
-    """Represents a file that has been sent as a Facebook attachment"""
-
-    #: Url where you can download the file
-    url = attr.ib(None)
-    #: Size of the file in bytes
-    size = attr.ib(None)
-    #: Name of the file
-    name = attr.ib(None)
-    #: Whether Facebook determines that this file may be harmful
-    is_malicious = attr.ib(None)
-
-    # Put here for backwards compatibility, so that the init argument order is preserved
-    uid = attr.ib(None)
-
-
-@attr.s(cmp=False)
-class AudioAttachment(Attachment):
-    """Represents an audio file that has been sent as a Facebook attachment"""
-
-    #: Name of the file
-    filename = attr.ib(None)
-    #: Url of the audio file
-    url = attr.ib(None)
-    #: Duration of the audioclip in milliseconds
-    duration = attr.ib(None)
-    #: Audio type
-    audio_type = attr.ib(None)
-
-    # Put here for backwards compatibility, so that the init argument order is preserved
-    uid = attr.ib(None)
-
-
-@attr.s(cmp=False, init=False)
-class ImageAttachment(Attachment):
-    """Represents an image that has been sent as a Facebook attachment
-
-    To retrieve the full image url, use: :func:`fbchat.Client.fetchImageUrl`, and pass
-    it the uid of the image attachment
-    """
-
-    #: The extension of the original image (eg. 'png')
-    original_extension = attr.ib(None)
-    #: Width of original image
-    width = attr.ib(None, converter=lambda x: None if x is None else int(x))
-    #: Height of original image
-    height = attr.ib(None, converter=lambda x: None if x is None else int(x))
-
-    #: Whether the image is animated
-    is_animated = attr.ib(None)
-
-    #: URL to a thumbnail of the image
-    thumbnail_url = attr.ib(None)
-
-    #: URL to a medium preview of the image
-    preview_url = attr.ib(None)
-    #: Width of the medium preview image
-    preview_width = attr.ib(None)
-    #: Height of the medium preview image
-    preview_height = attr.ib(None)
-
-    #: URL to a large preview of the image
-    large_preview_url = attr.ib(None)
-    #: Width of the large preview image
-    large_preview_width = attr.ib(None)
-    #: Height of the large preview image
-    large_preview_height = attr.ib(None)
-
-    #: URL to an animated preview of the image (eg. for gifs)
-    animated_preview_url = attr.ib(None)
-    #: Width of the animated preview image
-    animated_preview_width = attr.ib(None)
-    #: Height of the animated preview image
-    animated_preview_height = attr.ib(None)
-
-    def __init__(
-        self,
-        original_extension=None,
-        width=None,
-        height=None,
-        is_animated=None,
-        thumbnail_url=None,
-        preview=None,
-        large_preview=None,
-        animated_preview=None,
-        **kwargs
-    ):
-        super(ImageAttachment, self).__init__(**kwargs)
-        self.original_extension = original_extension
-        if width is not None:
-            width = int(width)
-        self.width = width
-        if height is not None:
-            height = int(height)
-        self.height = height
-        self.is_animated = is_animated
-        self.thumbnail_url = thumbnail_url
-
-        if preview is None:
-            preview = {}
-        self.preview_url = preview.get("uri")
-        self.preview_width = preview.get("width")
-        self.preview_height = preview.get("height")
-
-        if large_preview is None:
-            large_preview = {}
-        self.large_preview_url = large_preview.get("uri")
-        self.large_preview_width = large_preview.get("width")
-        self.large_preview_height = large_preview.get("height")
-
-        if animated_preview is None:
-            animated_preview = {}
-        self.animated_preview_url = animated_preview.get("uri")
-        self.animated_preview_width = animated_preview.get("width")
-        self.animated_preview_height = animated_preview.get("height")
-
-
-@attr.s(cmp=False, init=False)
-class VideoAttachment(Attachment):
-    """Represents a video that has been sent as a Facebook attachment"""
-
-    #: Size of the original video in bytes
-    size = attr.ib(None)
-    #: Width of original video
-    width = attr.ib(None)
-    #: Height of original video
-    height = attr.ib(None)
-    #: Length of video in milliseconds
-    duration = attr.ib(None)
-    #: URL to very compressed preview video
-    preview_url = attr.ib(None)
-
-    #: URL to a small preview image of the video
-    small_image_url = attr.ib(None)
-    #: Width of the small preview image
-    small_image_width = attr.ib(None)
-    #: Height of the small preview image
-    small_image_height = attr.ib(None)
-
-    #: URL to a medium preview image of the video
-    medium_image_url = attr.ib(None)
-    #: Width of the medium preview image
-    medium_image_width = attr.ib(None)
-    #: Height of the medium preview image
-    medium_image_height = attr.ib(None)
-
-    #: URL to a large preview image of the video
-    large_image_url = attr.ib(None)
-    #: Width of the large preview image
-    large_image_width = attr.ib(None)
-    #: Height of the large preview image
-    large_image_height = attr.ib(None)
-
-    def __init__(
-        self,
-        size=None,
-        width=None,
-        height=None,
-        duration=None,
-        preview_url=None,
-        small_image=None,
-        medium_image=None,
-        large_image=None,
-        **kwargs
-    ):
-        super(VideoAttachment, self).__init__(**kwargs)
-        self.size = size
-        self.width = width
-        self.height = height
-        self.duration = duration
-        self.preview_url = preview_url
-
-        if small_image is None:
-            small_image = {}
-        self.small_image_url = small_image.get("uri")
-        self.small_image_width = small_image.get("width")
-        self.small_image_height = small_image.get("height")
-
-        if medium_image is None:
-            medium_image = {}
-        self.medium_image_url = medium_image.get("uri")
-        self.medium_image_width = medium_image.get("width")
-        self.medium_image_height = medium_image.get("height")
-
-        if large_image is None:
-            large_image = {}
-        self.large_image_url = large_image.get("uri")
-        self.large_image_width = large_image.get("width")
-        self.large_image_height = large_image.get("height")

fbchat/_fix_module_metadata.py

@@ -0,0 +1,45 @@
+"""Everything in this module is taken from the excellent trio project.
+
+Having the public path in .__module__ attributes is important for:
+- exception names in printed tracebacks
+- ~sphinx :show-inheritance:~
+- deprecation warnings
+- pickle
+- probably other stuff
+"""
+
+import os
+
+
+def fixup_module_metadata(namespace):
+    def fix_one(qualname, name, obj):
+        # Custom extension, to handle classmethods, staticmethods and properties
+        if isinstance(obj, (classmethod, staticmethod)):
+            obj = obj.__func__
+        if isinstance(obj, property):
+            obj = obj.fget
+
+        mod = getattr(obj, "__module__", None)
+        if mod is not None and mod.startswith("fbchat."):
+            obj.__module__ = "fbchat"
+            # Modules, unlike everything else in Python, put fully-qualitied
+            # names into their __name__ attribute. We check for "." to avoid
+            # rewriting these.
+            if hasattr(obj, "__name__") and "." not in obj.__name__:
+                obj.__name__ = name
+                obj.__qualname__ = qualname
+            if isinstance(obj, type):
+                # Fix methods
+                for attr_name, attr_value in obj.__dict__.items():
+                    fix_one(objname + "." + attr_name, attr_name, attr_value)
+
+    for objname, obj in namespace.items():
+        if not objname.startswith("_"):  # ignore private attributes
+            fix_one(objname, objname, obj)
+
+
+# Allow disabling this when running Sphinx
+# This is done so that Sphinx autodoc can detect the file's source
+# TODO: Find a better way to detect when we're running Sphinx!
+if os.environ.get("_FBCHAT_DISABLE_FIX_MODULE_METADATA") == "1":
+    fixup_module_metadata = lambda namespace: None

fbchat/_graphql.py

@@ -1,10 +1,7 @@
-# -*- coding: UTF-8 -*-
-
-from __future__ import unicode_literals
 import json
 import re
-from .models import *
+from ._common import log
-from ._util import *
+from . import _util, _exception
 
 # Shameless copy from https://stackoverflow.com/a/8730674
 FLAGS = re.VERBOSE | re.MULTILINE | re.DOTALL
@@ -27,608 +24,31 @@ class ConcatJSONDecoder(json.JSONDecoder):
 # End shameless copy
 
 
-def graphql_color_to_enum(color):
+def queries_to_json(*queries):
-    if color is None:
-        return None
-    if not color:
-        return ThreadColor.MESSENGER_BLUE
-    color = color[2:]  # Strip the alpha value
-    color_value = "#{}".format(color.lower())
-    return enum_extend_if_invalid(ThreadColor, color_value)
-
-
-def get_customization_info(thread):
-    if thread is None or thread.get("customization_info") is None:
-        return {}
-    info = thread["customization_info"]
-
-    rtn = {
-        "emoji": info.get("emoji"),
-        "color": graphql_color_to_enum(info.get("outgoing_bubble_color")),
-    }
-    if (
-        thread.get("thread_type") == "GROUP"
-        or thread.get("is_group_thread")
-        or thread.get("thread_key", {}).get("thread_fbid")
-    ):
-        rtn["nicknames"] = {}
-        for k in info.get("participant_customizations", []):
-            rtn["nicknames"][k["participant_id"]] = k.get("nickname")
-    elif info.get("participant_customizations"):
-        uid = thread.get("thread_key", {}).get("other_user_id") or thread.get("id")
-        pc = info["participant_customizations"]
-        if len(pc) > 0:
-            if pc[0].get("participant_id") == uid:
-                rtn["nickname"] = pc[0].get("nickname")
-            else:
-                rtn["own_nickname"] = pc[0].get("nickname")
-        if len(pc) > 1:
-            if pc[1].get("participant_id") == uid:
-                rtn["nickname"] = pc[1].get("nickname")
-            else:
-                rtn["own_nickname"] = pc[1].get("nickname")
-    return rtn
-
-
-def graphql_to_sticker(s):
-    if not s:
-        return None
-    sticker = Sticker(uid=s["id"])
-    if s.get("pack"):
-        sticker.pack = s["pack"].get("id")
-    if s.get("sprite_image"):
-        sticker.is_animated = True
-        sticker.medium_sprite_image = s["sprite_image"].get("uri")
-        sticker.large_sprite_image = s["sprite_image_2x"].get("uri")
-        sticker.frames_per_row = s.get("frames_per_row")
-        sticker.frames_per_col = s.get("frames_per_column")
-        sticker.frame_rate = s.get("frame_rate")
-    sticker.url = s.get("url")
-    sticker.width = s.get("width")
-    sticker.height = s.get("height")
-    if s.get("label"):
-        sticker.label = s["label"]
-    return sticker
-
-
-def graphql_to_attachment(a):
-    _type = a["__typename"]
-    if _type in ["MessageImage", "MessageAnimatedImage"]:
-        return ImageAttachment(
-            original_extension=a.get("original_extension")
-            or (a["filename"].split("-")[0] if a.get("filename") else None),
-            width=a.get("original_dimensions", {}).get("width"),
-            height=a.get("original_dimensions", {}).get("height"),
-            is_animated=_type == "MessageAnimatedImage",
-            thumbnail_url=a.get("thumbnail", {}).get("uri"),
-            preview=a.get("preview") or a.get("preview_image"),
-            large_preview=a.get("large_preview"),
-            animated_preview=a.get("animated_image"),
-            uid=a.get("legacy_attachment_id"),
-        )
-    elif _type == "MessageVideo":
-        return VideoAttachment(
-            width=a.get("original_dimensions", {}).get("width"),
-            height=a.get("original_dimensions", {}).get("height"),
-            duration=a.get("playable_duration_in_ms"),
-            preview_url=a.get("playable_url"),
-            small_image=a.get("chat_image"),
-            medium_image=a.get("inbox_image"),
-            large_image=a.get("large_image"),
-            uid=a.get("legacy_attachment_id"),
-        )
-    elif _type == "MessageAudio":
-        return AudioAttachment(
-            filename=a.get("filename"),
-            url=a.get("playable_url"),
-            duration=a.get("playable_duration_in_ms"),
-            audio_type=a.get("audio_type"),
-        )
-    elif _type == "MessageFile":
-        return FileAttachment(
-            url=a.get("url"),
-            name=a.get("filename"),
-            is_malicious=a.get("is_malicious"),
-            uid=a.get("message_file_fbid"),
-        )
-    else:
-        return Attachment(uid=a.get("legacy_attachment_id"))
-
-
-def graphql_to_extensible_attachment(a):
-    story = a.get("story_attachment")
-    if story:
-        target = story.get("target")
-        if target:
-            _type = target["__typename"]
-            if _type == "MessageLocation":
-                url = story.get("url")
-                address = get_url_parameter(get_url_parameter(url, "u"), "where1")
-                try:
-                    latitude, longitude = [float(x) for x in address.split(", ")]
-                    address = None
-                except ValueError:
-                    latitude, longitude = None, None
-                rtn = LocationAttachment(
-                    uid=int(story["deduplication_key"]),
-                    latitude=latitude,
-                    longitude=longitude,
-                    address=address,
-                )
-                media = story.get("media")
-                if media and media.get("image"):
-                    image = media["image"]
-                    rtn.image_url = image.get("uri")
-                    rtn.image_width = image.get("width")
-                    rtn.image_height = image.get("height")
-                rtn.url = url
-                return rtn
-            elif _type == "MessageLiveLocation":
-                rtn = LiveLocationAttachment(
-                    uid=int(story["target"]["live_location_id"]),
-                    latitude=story["target"]["coordinate"]["latitude"]
-                    if story["target"].get("coordinate")
-                    else None,
-                    longitude=story["target"]["coordinate"]["longitude"]
-                    if story["target"].get("coordinate")
-                    else None,
-                    name=story["title_with_entities"]["text"],
-                    expiration_time=story["target"].get("expiration_time"),
-                    is_expired=story["target"].get("is_expired"),
-                )
-                media = story.get("media")
-                if media and media.get("image"):
-                    image = media["image"]
-                    rtn.image_url = image.get("uri")
-                    rtn.image_width = image.get("width")
-                    rtn.image_height = image.get("height")
-                rtn.url = story.get("url")
-                return rtn
-            elif _type in ["ExternalUrl", "Story"]:
-                url = story.get("url")
-                rtn = ShareAttachment(
-                    uid=a.get("legacy_attachment_id"),
-                    author=story["target"]["actors"][0]["id"]
-                    if story["target"].get("actors")
-                    else None,
-                    url=url,
-                    original_url=get_url_parameter(url, "u")
-                    if "/l.php?u=" in url
-                    else url,
-                    title=story["title_with_entities"].get("text"),
-                    description=story["description"].get("text")
-                    if story.get("description")
-                    else None,
-                    source=story["source"].get("text"),
-                    attachments=[
-                        graphql_to_subattachment(attachment)
-                        for attachment in story.get("subattachments")
-                    ],
-                )
-                media = story.get("media")
-                if media and media.get("image"):
-                    image = media["image"]
-                    rtn.image_url = image.get("uri")
-                    rtn.original_image_url = (
-                        get_url_parameter(rtn.image_url, "url")
-                        if "/safe_image.php" in rtn.image_url
-                        else rtn.image_url
-                    )
-                    rtn.image_width = image.get("width")
-                    rtn.image_height = image.get("height")
-                return rtn
-        else:
-            return UnsentMessage(uid=a.get("legacy_attachment_id"))
-
-
-def graphql_to_subattachment(a):
-    _type = a["target"]["__typename"]
-    if _type == "Video":
-        media = a["media"]
-        return VideoAttachment(
-            duration=media.get("playable_duration_in_ms"),
-            preview_url=media.get("playable_url"),
-            medium_image=media.get("image"),
-            uid=a["target"].get("video_id"),
-        )
-
-
-def graphql_to_live_location(a):
-    return LiveLocationAttachment(
-        uid=a["id"],
-        latitude=a["coordinate"]["latitude"] / (10 ** 8)
-        if not a.get("stopReason")
-        else None,
-        longitude=a["coordinate"]["longitude"] / (10 ** 8)
-        if not a.get("stopReason")
-        else None,
-        name=a.get("locationTitle"),
-        expiration_time=a["expirationTime"],
-        is_expired=bool(a.get("stopReason")),
-    )
-
-
-def graphql_to_poll(a):
-    rtn = Poll(
-        title=a.get("title") if a.get("title") else a.get("text"),
-        options=[graphql_to_poll_option(m) for m in a.get("options")],
-    )
-    rtn.uid = int(a["id"])
-    rtn.options_count = a.get("total_count")
-    return rtn
-
-
-def graphql_to_poll_option(a):
-    if a.get("viewer_has_voted") is None:
-        vote = None
-    elif isinstance(a["viewer_has_voted"], bool):
-        vote = a["viewer_has_voted"]
-    else:
-        vote = a["viewer_has_voted"] == "true"
-    rtn = PollOption(text=a.get("text"), vote=vote)
-    rtn.uid = int(a["id"])
-    rtn.voters = (
-        [m.get("node").get("id") for m in a.get("voters").get("edges")]
-        if isinstance(a.get("voters"), dict)
-        else a.get("voters")
-    )
-    rtn.votes_count = (
-        a.get("voters").get("count")
-        if isinstance(a.get("voters"), dict)
-        else a.get("total_count")
-    )
-    return rtn
-
-
-def graphql_to_plan(a):
-    if a.get("event_members"):
-        rtn = Plan(
-            time=a.get("event_time"),
-            title=a.get("title"),
-            location=a.get("location_name"),
-        )
-        if a.get("location_id") != 0:
-            rtn.location_id = str(a.get("location_id"))
-        rtn.uid = a.get("oid")
-        rtn.author_id = a.get("creator_id")
-        guests = a.get("event_members")
-        rtn.going = [uid for uid in guests if guests[uid] == "GOING"]
-        rtn.declined = [uid for uid in guests if guests[uid] == "DECLINED"]
-        rtn.invited = [uid for uid in guests if guests[uid] == "INVITED"]
-        return rtn
-    elif a.get("id") is None:
-        rtn = Plan(
-            time=a.get("event_time"),
-            title=a.get("event_title"),
-            location=a.get("event_location_name"),
-            location_id=a.get("event_location_id"),
-        )
-        rtn.uid = a.get("event_id")
-        rtn.author_id = a.get("event_creator_id")
-        guests = json.loads(a.get("guest_state_list"))
-    else:
-        rtn = Plan(
-            time=a.get("time"),
-            title=a.get("event_title"),
-            location=a.get("location_name"),
-        )
-        rtn.uid = a.get("id")
-        rtn.author_id = a.get("lightweight_event_creator").get("id")
-        guests = a.get("event_reminder_members").get("edges")
-    rtn.going = [
-        m.get("node").get("id") for m in guests if m.get("guest_list_state") == "GOING"
-    ]
-    rtn.declined = [
-        m.get("node").get("id")
-        for m in guests
-        if m.get("guest_list_state") == "DECLINED"
-    ]
-    rtn.invited = [
-        m.get("node").get("id")
-        for m in guests
-        if m.get("guest_list_state") == "INVITED"
-    ]
-    return rtn
-
-
-def graphql_to_quick_reply(q, is_response=False):
-    data = dict()
-    _type = q.get("content_type").lower()
-    if q.get("payload"):
-        data["payload"] = q["payload"]
-    if q.get("data"):
-        data["data"] = q["data"]
-    if q.get("image_url") and _type is not QuickReplyLocation._type:
-        data["image_url"] = q["image_url"]
-    data["is_response"] = is_response
-    if _type == QuickReplyText._type:
-        if q.get("title") is not None:
-            data["title"] = q["title"]
-        rtn = QuickReplyText(**data)
-    elif _type == QuickReplyLocation._type:
-        rtn = QuickReplyLocation(**data)
-    elif _type == QuickReplyPhoneNumber._type:
-        rtn = QuickReplyPhoneNumber(**data)
-    elif _type == QuickReplyEmail._type:
-        rtn = QuickReplyEmail(**data)
-    return rtn
-
-
-def graphql_to_message(message):
-    if message.get("message_sender") is None:
-        message["message_sender"] = {}
-    if message.get("message") is None:
-        message["message"] = {}
-    rtn = Message(
-        text=message.get("message").get("text"),
-        mentions=[
-            Mention(
-                m.get("entity", {}).get("id"),
-                offset=m.get("offset"),
-                length=m.get("length"),
-            )
-            for m in message.get("message").get("ranges", [])
-        ],
-        emoji_size=get_emojisize_from_tags(message.get("tags_list")),
-        sticker=graphql_to_sticker(message.get("sticker")),
-    )
-    rtn.uid = str(message.get("message_id"))
-    rtn.author = str(message.get("message_sender").get("id"))
-    rtn.timestamp = message.get("timestamp_precise")
-    rtn.unsent = False
-    if message.get("unread") is not None:
-        rtn.is_read = not message["unread"]
-    rtn.reactions = {
-        str(r["user"]["id"]): enum_extend_if_invalid(MessageReaction, r["reaction"])
-        for r in message.get("message_reactions")
-    }
-    if message.get("blob_attachments") is not None:
-        rtn.attachments = [
-            graphql_to_attachment(attachment)
-            for attachment in message["blob_attachments"]
-        ]
-    if message.get("platform_xmd_encoded"):
-        quick_replies = json.loads(message["platform_xmd_encoded"]).get("quick_replies")
-        if isinstance(quick_replies, list):
-            rtn.quick_replies = [graphql_to_quick_reply(q) for q in quick_replies]
-        elif isinstance(quick_replies, dict):
-            rtn.quick_replies = [
-                graphql_to_quick_reply(quick_replies, is_response=True)
-            ]
-    if message.get("extensible_attachment") is not None:
-        attachment = graphql_to_extensible_attachment(message["extensible_attachment"])
-        if isinstance(attachment, UnsentMessage):
-            rtn.unsent = True
-        elif attachment:
-            rtn.attachments.append(attachment)
-    if message.get("replied_to_message") is not None:
-        rtn.replied_to = graphql_to_message(message["replied_to_message"]["message"])
-    return rtn
-
-
-def graphql_to_message_reply(message):
-    rtn = Message(
-        text=message.get("body"),
-        mentions=[
-            Mention(m.get("i"), offset=m.get("o"), length=m.get("l"))
-            for m in json.loads(message.get("data", {}).get("prng", "[]"))
-        ],
-        emoji_size=get_emojisize_from_tags(message["messageMetadata"].get("tags")),
-    )
-    metadata = message.get("messageMetadata", {})
-    rtn.uid = metadata.get("messageId")
-    rtn.author = str(metadata.get("actorFbId"))
-    rtn.timestamp = metadata.get("timestamp")
-    rtn.unsent = False
-    if message.get("data", {}).get("platform_xmd"):
-        quick_replies = json.loads(message["data"]["platform_xmd"]).get("quick_replies")
-        if isinstance(quick_replies, list):
-            rtn.quick_replies = [graphql_to_quick_reply(q) for q in quick_replies]
-        elif isinstance(quick_replies, dict):
-            rtn.quick_replies = [
-                graphql_to_quick_reply(quick_replies, is_response=True)
-            ]
-    if message.get("attachments") is not None:
-        for attachment in message["attachments"]:
-            attachment = json.loads(attachment["mercuryJSON"])
-            if attachment.get("blob_attachment"):
-                rtn.attachments.append(
-                    graphql_to_attachment(attachment["blob_attachment"])
-                )
-            if attachment.get("extensible_attachment"):
-                extensible_attachment = graphql_to_extensible_attachment(
-                    attachment["extensible_attachment"]
-                )
-                if isinstance(extensible_attachment, UnsentMessage):
-                    rtn.unsent = True
-                else:
-                    rtn.attachments.append(extensible_attachment)
-            if attachment.get("sticker_attachment"):
-                rtn.sticker = graphql_to_sticker(attachment["sticker_attachment"])
-    return rtn
-
-
-def graphql_to_user(user):
-    if user.get("profile_picture") is None:
-        user["profile_picture"] = {}
-    c_info = get_customization_info(user)
-    plan = None
-    if user.get("event_reminders"):
-        plan = (
-            graphql_to_plan(user["event_reminders"]["nodes"][0])
-            if user["event_reminders"].get("nodes")
-            else None
-        )
-    return User(
-        user["id"],
-        url=user.get("url"),
-        first_name=user.get("first_name"),
-        last_name=user.get("last_name"),
-        is_friend=user.get("is_viewer_friend"),
-        gender=GENDERS.get(user.get("gender")),
-        affinity=user.get("affinity"),
-        nickname=c_info.get("nickname"),
-        color=c_info.get("color"),
-        emoji=c_info.get("emoji"),
-        own_nickname=c_info.get("own_nickname"),
-        photo=user["profile_picture"].get("uri"),
-        name=user.get("name"),
-        message_count=user.get("messages_count"),
-        plan=plan,
-    )
-
-
-def graphql_to_thread(thread):
-    if thread["thread_type"] == "GROUP":
-        return graphql_to_group(thread)
-    elif thread["thread_type"] == "ONE_TO_ONE":
-        if thread.get("big_image_src") is None:
-            thread["big_image_src"] = {}
-        c_info = get_customization_info(thread)
-        participants = [
-            node["messaging_actor"] for node in thread["all_participants"]["nodes"]
-        ]
-        user = next(
-            p for p in participants if p["id"] == thread["thread_key"]["other_user_id"]
-        )
-        last_message_timestamp = None
-        if "last_message" in thread:
-            last_message_timestamp = thread["last_message"]["nodes"][0][
-                "timestamp_precise"
-            ]
-
-        first_name = user.get("short_name")
-        if first_name is None:
-            last_name = None
-        else:
-            last_name = user.get("name").split(first_name, 1).pop().strip()
-
-        plan = None
-        if thread.get("event_reminders"):
-            plan = (
-                graphql_to_plan(thread["event_reminders"]["nodes"][0])
-                if thread["event_reminders"].get("nodes")
-                else None
-            )
-
-        return User(
-            user["id"],
-            url=user.get("url"),
-            name=user.get("name"),
-            first_name=first_name,
-            last_name=last_name,
-            is_friend=user.get("is_viewer_friend"),
-            gender=GENDERS.get(user.get("gender")),
-            affinity=user.get("affinity"),
-            nickname=c_info.get("nickname"),
-            color=c_info.get("color"),
-            emoji=c_info.get("emoji"),
-            own_nickname=c_info.get("own_nickname"),
-            photo=user["big_image_src"].get("uri"),
-            message_count=thread.get("messages_count"),
-            last_message_timestamp=last_message_timestamp,
-            plan=plan,
-        )
-    else:
-        raise FBchatException(
-            "Unknown thread type: {}, with data: {}".format(
-                thread.get("thread_type"), thread
-            )
-        )
-
-
-def graphql_to_group(group):
-    if group.get("image") is None:
-        group["image"] = {}
-    c_info = get_customization_info(group)
-    last_message_timestamp = None
-    if "last_message" in group:
-        last_message_timestamp = group["last_message"]["nodes"][0]["timestamp_precise"]
-    plan = None
-    if group.get("event_reminders"):
-        plan = (
-            graphql_to_plan(group["event_reminders"]["nodes"][0])
-            if group["event_reminders"].get("nodes")
-            else None
-        )
-    return Group(
-        group["thread_key"]["thread_fbid"],
-        participants=set(
-            [
-                node["messaging_actor"]["id"]
-                for node in group["all_participants"]["nodes"]
-            ]
-        ),
-        nicknames=c_info.get("nicknames"),
-        color=c_info.get("color"),
-        emoji=c_info.get("emoji"),
-        admins=set([node.get("id") for node in group.get("thread_admins")]),
-        approval_mode=bool(group.get("approval_mode"))
-        if group.get("approval_mode") is not None
-        else None,
-        approval_requests=set(
-            node["requester"]["id"] for node in group["group_approval_queue"]["nodes"]
-        )
-        if group.get("group_approval_queue")
-        else None,
-        join_link=group["joinable_mode"].get("link"),
-        photo=group["image"].get("uri"),
-        name=group.get("name"),
-        message_count=group.get("messages_count"),
-        last_message_timestamp=last_message_timestamp,
-        plan=plan,
-    )
-
-
-def graphql_to_page(page):
-    if page.get("profile_picture") is None:
-        page["profile_picture"] = {}
-    if page.get("city") is None:
-        page["city"] = {}
-    plan = None
-    if page.get("event_reminders"):
-        plan = (
-            graphql_to_plan(page["event_reminders"]["nodes"][0])
-            if page["event_reminders"].get("nodes")
-            else None
-        )
-    return Page(
-        page["id"],
-        url=page.get("url"),
-        city=page.get("city").get("name"),
-        category=page.get("category_type"),
-        photo=page["profile_picture"].get("uri"),
-        name=page.get("name"),
-        message_count=page.get("messages_count"),
-        plan=plan,
-    )
-
-
-def graphql_queries_to_json(*queries):
     """
     Queries should be a list of GraphQL objects
     """
     rtn = {}
     for i, query in enumerate(queries):
-        rtn["q{}".format(i)] = query.value
+        rtn["q{}".format(i)] = query
-    return json.dumps(rtn)
+    return _util.json_minimal(rtn)
 
 
-def graphql_response_to_json(content):
+def response_to_json(text):
-    content = strip_to_json(content)  # Usually only needed in some error cases
+    text = _util.strip_json_cruft(text)  # Usually only needed in some error cases
     try:
-        j = json.loads(content, cls=ConcatJSONDecoder)
+        j = json.loads(text, cls=ConcatJSONDecoder)
-    except Exception:
+    except Exception as e:
-        raise FBchatException("Error while parsing JSON: {}".format(repr(content)))
+        raise _exception.ParseError("Error while parsing JSON", data=text) from e
 
     rtn = [None] * (len(j))
     for x in j:
         if "error_results" in x:
             del rtn[-1]
             continue
-        check_json(x)
+        _exception.handle_payload_error(x)
         [(key, value)] = x.items()
-        check_json(value)
+        _exception.handle_graphql_errors(value)
         if "response" in value:
             rtn[int(key[1:])] = value["response"]
         else:
@@ -639,16 +59,21 @@ def graphql_response_to_json(content):
     return rtn
 
 
-class GraphQL(object):
+def from_query(query, params):
-    def __init__(self, query=None, doc_id=None, params=None):
+    return {"priority": 0, "q": query, "query_params": params}
-        if params is None:
+
-            params = {}
+
-        if query is not None:
+def from_query_id(query_id, params):
-            self.value = {"priority": 0, "q": query, "query_params": params}
+    return {"query_id": query_id, "query_params": params}
-        elif doc_id is not None:
+
-            self.value = {"doc_id": doc_id, "query_params": params}
+
-        else:
+def from_doc(doc, params):
-            raise FBchatUserError("A query or doc_id must be specified")
+    return {"doc": doc, "query_params": params}
+
+
+def from_doc_id(doc_id, params):
+    return {"doc_id": doc_id, "query_params": params}
+
 
 FRAGMENT_USER = """
 QueryFragment User: User {
@@ -679,6 +104,7 @@ class GraphQL(object):
     all_participants {
         nodes {
             messaging_actor {
+                __typename,
                 id
             }
         }

fbchat/_group.py

@@ -1,74 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-from ._thread import ThreadType, Thread
-
-
-@attr.s(cmp=False, init=False)
-class Group(Thread):
-    """Represents a Facebook group. Inherits `Thread`"""
-
-    #: Unique list (set) of the group thread's participant user IDs
-    participants = attr.ib(factory=set, converter=lambda x: set() if x is None else x)
-    #: A dict, containing user nicknames mapped to their IDs
-    nicknames = attr.ib(factory=dict, converter=lambda x: {} if x is None else x)
-    #: A :class:`ThreadColor`. The groups's message color
-    color = attr.ib(None)
-    #: The groups's default emoji
-    emoji = attr.ib(None)
-    # Set containing user IDs of thread admins
-    admins = attr.ib(factory=set, converter=lambda x: set() if x is None else x)
-    # True if users need approval to join
-    approval_mode = attr.ib(None)
-    # Set containing user IDs requesting to join
-    approval_requests = attr.ib(
-        factory=set, converter=lambda x: set() if x is None else x
-    )
-    # Link for joining group
-    join_link = attr.ib(None)
-
-    def __init__(
-        self,
-        uid,
-        participants=None,
-        nicknames=None,
-        color=None,
-        emoji=None,
-        admins=None,
-        approval_mode=None,
-        approval_requests=None,
-        join_link=None,
-        privacy_mode=None,
-        **kwargs
-    ):
-        super(Group, self).__init__(ThreadType.GROUP, uid, **kwargs)
-        if participants is None:
-            participants = set()
-        self.participants = participants
-        if nicknames is None:
-            nicknames = []
-        self.nicknames = nicknames
-        self.color = color
-        self.emoji = emoji
-        if admins is None:
-            admins = set()
-        self.admins = admins
-        self.approval_mode = approval_mode
-        if approval_requests is None:
-            approval_requests = set()
-        self.approval_requests = approval_requests
-        self.join_link = join_link
-
-
-@attr.s(cmp=False, init=False)
-class Room(Group):
-    """Deprecated. Use :class:`Group` instead"""
-
-    # True is room is not discoverable
-    privacy_mode = attr.ib(None)
-
-    def __init__(self, uid, privacy_mode=None, **kwargs):
-        super(Room, self).__init__(uid, **kwargs)
-        self.type = ThreadType.ROOM
-        self.privacy_mode = privacy_mode

fbchat/_listen.py

@@ -0,0 +1,407 @@
+import attr
+import random
+import paho.mqtt.client
+import requests
+from ._common import log, kw_only
+from . import _util, _exception, _session, _graphql, _events
+
+from typing import Iterable, Optional, Mapping, List
+
+
+HOST = "edge-chat.messenger.com"
+
+TOPICS = [
+    # Things that happen in chats (e.g. messages)
+    "/t_ms",
+    # Group typing notifications
+    "/thread_typing",
+    # Private chat typing notifications
+    "/orca_typing_notifications",
+    # Active notifications
+    "/orca_presence",
+    # Other notifications not related to chats (e.g. friend requests)
+    "/legacy_web",
+    # Facebook's continuous error reporting/logging?
+    "/br_sr",
+    # Response to /br_sr
+    "/sr_res",
+    # Data about user-to-user calls
+    # TODO: Investigate the response from this! (A bunch of binary data)
+    # "/t_rtc",
+    # TODO: Find out what this does!
+    # TODO: Investigate the response from this! (A bunch of binary data)
+    # "/t_p",
+    # TODO: Find out what this does!
+    "/webrtc",
+    # TODO: Find out what this does!
+    "/onevc",
+    # TODO: Find out what this does!
+    "/notify_disconnect",
+    # Old, no longer active topics
+    # These are here just in case something interesting pops up
+    "/inbox",
+    "/mercury",
+    "/messaging_events",
+    "/orca_message_notifications",
+    "/pp",
+    "/webrtc_response",
+]
+
+
+def get_cookie_header(session: requests.Session, url: str) -> str:
+    """Extract a cookie header from a requests session."""
+    # The cookies are extracted this way to make sure they're escaped correctly
+    return requests.cookies.get_cookie_header(
+        session.cookies, requests.Request("GET", url),
+    )
+
+
+def generate_session_id() -> int:
+    """Generate a random session ID between 1 and 9007199254740991."""
+    return random.randint(1, 2 ** 53)
+
+
+def mqtt_factory() -> paho.mqtt.client.Client:
+    # Configure internal MQTT handler
+    mqtt = paho.mqtt.client.Client(
+        client_id="mqttwsclient",
+        clean_session=True,
+        protocol=paho.mqtt.client.MQTTv31,
+        transport="websockets",
+    )
+    mqtt.enable_logger()
+    # mqtt.max_inflight_messages_set(20)  # The rest will get queued
+    # mqtt.max_queued_messages_set(0)  # Unlimited messages can be queued
+    # mqtt.message_retry_set(20)  # Retry sending for at least 20 seconds
+    # mqtt.reconnect_delay_set(min_delay=1, max_delay=120)
+    mqtt.tls_set()
+    mqtt.connect_async(HOST, 443, keepalive=10)
+    return mqtt
+
+
+def fetch_sequence_id(session: _session.Session) -> int:
+    """Fetch sequence ID."""
+    params = {
+        "limit": 0,
+        "tags": ["INBOX"],
+        "before": None,
+        "includeDeliveryReceipts": False,
+        "includeSeqID": True,
+    }
+    log.debug("Fetching MQTT sequence ID")
+    # Same doc id as in `Client.fetch_threads`
+    (j,) = session._graphql_requests(_graphql.from_doc_id("1349387578499440", params))
+    sequence_id = j["viewer"]["message_threads"]["sync_sequence_id"]
+    if not sequence_id:
+        raise _exception.NotLoggedIn("Failed fetching sequence id")
+    return int(sequence_id)
+
+
+@attr.s(slots=True, kw_only=kw_only, eq=False)
+class Listener:
+    """Listen to incoming Facebook events.
+
+    Initialize a connection to the Facebook MQTT service.
+
+    Args:
+        session: The session to use when making requests.
+        chat_on: Whether ...
+        foreground: Whether ...
+
+    Example:
+        >>> listener = fbchat.Listener(session, chat_on=True, foreground=True)
+    """
+
+    session = attr.ib(type=_session.Session)
+    _chat_on = attr.ib(type=bool)
+    _foreground = attr.ib(type=bool)
+    _mqtt = attr.ib(factory=mqtt_factory, type=paho.mqtt.client.Client)
+    _sync_token = attr.ib(None, type=Optional[str])
+    _sequence_id = attr.ib(None, type=Optional[int])
+    _tmp_events = attr.ib(factory=list, type=List[_events.Event])
+
+    def __attrs_post_init__(self):
+        # Configure callbacks
+        self._mqtt.on_message = self._on_message_handler
+        self._mqtt.on_connect = self._on_connect_handler
+
+    def _handle_ms(self, j):
+        """Handle /t_ms special logic.
+
+        Returns whether to continue parsing the message.
+        """
+        # TODO: Merge this with the parsing in _events
+
+        # Update sync_token when received
+        # This is received in the first message after we've created a messenger
+        # sync queue.
+        if "syncToken" in j and "firstDeltaSeqId" in j:
+            self._sync_token = j["syncToken"]
+            self._sequence_id = j["firstDeltaSeqId"]
+            return False
+
+        if "errorCode" in j:
+            error = j["errorCode"]
+            # TODO: 'F\xfa\x84\x8c\x85\xf8\xbc-\x88 FB_PAGES_INSUFFICIENT_PERMISSION\x00'
+            if error in ("ERROR_QUEUE_NOT_FOUND", "ERROR_QUEUE_OVERFLOW"):
+                # ERROR_QUEUE_NOT_FOUND means that the queue was deleted, since too
+                # much time passed, or that it was simply missing
+                # ERROR_QUEUE_OVERFLOW means that the sequence id was too small, so
+                # the desired events could not be retrieved
+                log.error(
+                    "The MQTT listener was disconnected for too long,"
+                    " events may have been lost"
+                )
+                # TODO: Find a way to tell the user that they may now be missing events
+                self._sync_token = None
+                self._sequence_id = None
+                return False
+            log.error("MQTT error code %s received", error)
+            return False
+
+        # Update last sequence id
+        # Except for the two cases above, this is always received
+        self._sequence_id = j["lastIssuedSeqId"]
+        return True
+
+    def _on_message_handler(self, client, userdata, message):
+        # Parse payload JSON
+        try:
+            j = _util.parse_json(message.payload.decode("utf-8"))
+        except (_exception.FacebookError, UnicodeDecodeError):
+            log.debug(message.payload)
+            log.exception("Failed parsing MQTT data on %s as JSON", message.topic)
+            return
+
+        log.debug("MQTT payload: %s, %s", message.topic, j)
+
+        if message.topic == "/t_ms":
+            if not self._handle_ms(j):
+                return
+
+        try:
+            # TODO: Don't handle this in a callback
+            self._tmp_events = list(
+                _events.parse_events(self.session, message.topic, j)
+            )
+        except _exception.ParseError:
+            log.exception("Failed parsing MQTT data")
+
+    def _on_connect_handler(self, client, userdata, flags, rc):
+        if rc == 21:
+            raise _exception.FacebookError(
+                "Failed connecting. Maybe your cookies are wrong?"
+            )
+        if rc != 0:
+            err = paho.mqtt.client.connack_string(rc)
+            log.error("MQTT Connection Error: %s", err)
+            return  # Don't try to send publish if the connection failed
+
+        self._messenger_queue_publish()
+
+    def _messenger_queue_publish(self):
+        # configure receiving messages.
+        payload = {
+            "sync_api_version": 10,
+            "max_deltas_able_to_process": 1000,
+            "delta_batch_size": 500,
+            "encoding": "JSON",
+            "entity_fbid": self.session.user.id,
+        }
+
+        # If we don't have a sync_token, create a new messenger queue
+        # This is done so that across reconnects, if we've received a sync token, we
+        # SHOULD receive a piece of data in /t_ms exactly once!
+        if self._sync_token is None:
+            topic = "/messenger_sync_create_queue"
+            payload["initial_titan_sequence_id"] = str(self._sequence_id)
+            payload["device_params"] = None
+        else:
+            topic = "/messenger_sync_get_diffs"
+            payload["last_seq_id"] = str(self._sequence_id)
+            payload["sync_token"] = self._sync_token
+
+        self._mqtt.publish(topic, _util.json_minimal(payload), qos=1)
+
+    def _configure_connect_options(self):
+        # Generate a new session ID on each reconnect
+        session_id = generate_session_id()
+
+        username = {
+            # The user ID
+            "u": self.session.user.id,
+            # Session ID
+            "s": session_id,
+            # Active status setting
+            "chat_on": self._chat_on,
+            # foreground_state - Whether the window is focused
+            "fg": self._foreground,
+            # Can be any random ID
+            "d": self.session._client_id,
+            # Application ID, taken from facebook.com
+            "aid": 219994525426954,
+            # MQTT extension by FB, allows making a SUBSCRIBE while CONNECTing
+            "st": TOPICS,
+            # MQTT extension by FB, allows making a PUBLISH while CONNECTing
+            # Using this is more efficient, but the same can be acheived with:
+            #     def on_connect(*args):
+            #         mqtt.publish(topic, payload, qos=1)
+            #     mqtt.on_connect = on_connect
+            # TODO: For some reason this doesn't work!
+            "pm": [
+                # {
+                #     "topic": topic,
+                #     "payload": payload,
+                #     "qos": 1,
+                #     "messageId": 65536,
+                # }
+            ],
+            # Unknown parameters
+            "cp": 3,
+            "ecp": 10,
+            "ct": "websocket",
+            "mqtt_sid": "",
+            "dc": "",
+            "no_auto_fg": True,
+            "gas": None,
+            "pack": [],
+        }
+
+        self._mqtt.username_pw_set(_util.json_minimal(username))
+
+        headers = {
+            "Cookie": get_cookie_header(
+                self.session._session, "https://edge-chat.messenger.com/chat"
+            ),
+            "User-Agent": self.session._session.headers["User-Agent"],
+            "Origin": "https://www.messenger.com",
+            "Host": HOST,
+        }
+
+        # TODO: Is region (lla | atn | odn | others?) important?
+        self._mqtt.ws_set_options(
+            path="/chat?sid={}".format(session_id), headers=headers
+        )
+
+    def _reconnect(self) -> bool:
+        # Try reconnecting
+        self._configure_connect_options()
+        try:
+            self._mqtt.reconnect()
+            return True
+        except (
+            # Taken from .loop_forever
+            paho.mqtt.client.socket.error,
+            OSError,
+            paho.mqtt.client.WebsocketConnectionError,
+        ) as e:
+            log.debug("MQTT reconnection failed: %s", e)
+            # Wait before reconnecting
+            self._mqtt._reconnect_wait()
+            return False
+
+    def listen(self) -> Iterable[_events.Event]:
+        """Run the listening loop continually.
+
+        This is a blocking call, that will yield events as they arrive.
+
+        This will automatically reconnect on errors, except if the errors are one of
+        `PleaseRefresh` or `NotLoggedIn`.
+
+        Example:
+            Print events continually.
+
+            >>> for event in listener.listen():
+            ...     print(event)
+        """
+        if self._sequence_id is None:
+            self._sequence_id = fetch_sequence_id(self.session)
+
+        # Make sure we're connected
+        while not self._reconnect():
+            pass
+
+        yield _events.Connect()
+
+        while True:
+            rc = self._mqtt.loop(timeout=1.0)
+
+            # The sequence ID was reset in _handle_ms
+            # TODO: Signal to the user that they should reload their data!
+            if self._sequence_id is None:
+                self._sequence_id = fetch_sequence_id(self.session)
+                self._messenger_queue_publish()
+
+            # If disconnect() has been called
+            # Beware, internal API, may have to change this to something more stable!
+            if self._mqtt._state == paho.mqtt.client.mqtt_cs_disconnecting:
+                break  # Stop listening
+
+            if rc != paho.mqtt.client.MQTT_ERR_SUCCESS:
+                # If known/expected error
+                if rc == paho.mqtt.client.MQTT_ERR_CONN_LOST:
+                    yield _events.Disconnect(reason="Connection lost, retrying")
+                elif rc == paho.mqtt.client.MQTT_ERR_NOMEM:
+                    # This error is wrongly classified
+                    # See https://github.com/eclipse/paho.mqtt.python/issues/340
+                    yield _events.Disconnect(reason="Connection error, retrying")
+                elif rc == paho.mqtt.client.MQTT_ERR_CONN_REFUSED:
+                    raise _exception.NotLoggedIn("MQTT connection refused")
+                else:
+                    err = paho.mqtt.client.error_string(rc)
+                    log.error("MQTT Error: %s", err)
+                    reason = "MQTT Error: {}, retrying".format(err)
+                    yield _events.Disconnect(reason=reason)
+
+                while not self._reconnect():
+                    pass
+
+                yield _events.Connect()
+
+            if self._tmp_events:
+                yield from self._tmp_events
+                self._tmp_events = []
+
+    def disconnect(self) -> None:
+        """Disconnect the MQTT listener.
+
+        Can be called while listening, which will stop the listening loop.
+
+        The `Listener` object should not be used after this is called!
+
+        Example:
+            Stop the listener when receiving a message with the text "/stop"
+
+            >>> for event in listener.listen():
+            ...     if isinstance(event, fbchat.MessageEvent):
+            ...         if event.message.text == "/stop":
+            ...             listener.disconnect()  # Almost the same "break"
+        """
+        self._mqtt.disconnect()
+
+    def set_foreground(self, value: bool) -> None:
+        """Set the ``foreground`` value while listening."""
+        # TODO: Document what this actually does!
+        payload = _util.json_minimal({"foreground": value})
+        info = self._mqtt.publish("/foreground_state", payload=payload, qos=1)
+        self._foreground = value
+        # TODO: We can't wait for this, since the loop is running within the same thread
+        # info.wait_for_publish()
+
+    def set_chat_on(self, value: bool) -> None:
+        """Set the ``chat_on`` value while listening."""
+        # TODO: Document what this actually does!
+        # TODO: Is this the right request to make?
+        data = {"make_user_available_when_in_foreground": value}
+        payload = _util.json_minimal(data)
+        info = self._mqtt.publish("/set_client_settings", payload=payload, qos=1)
+        self._chat_on = value
+        # TODO: We can't wait for this, since the loop is running within the same thread
+        # info.wait_for_publish()
+
+    # def send_additional_contacts(self, additional_contacts):
+    #     payload = _util.json_minimal({"additional_contacts": additional_contacts})
+    #     info = self._mqtt.publish("/send_additional_contacts", payload=payload, qos=1)
+    #
+    # def browser_close(self):
+    #     info = self._mqtt.publish("/browser_close", payload=b"{}", qos=1)

fbchat/_location.py

@@ -1,48 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-from ._attachment import Attachment
-
-
-@attr.s(cmp=False)
-class LocationAttachment(Attachment):
-    """Represents a user location
-
-    Latitude and longitude OR address is provided by Facebook
-    """
-
-    #: Latitude of the location
-    latitude = attr.ib(None)
-    #: Longitude of the location
-    longitude = attr.ib(None)
-    #: URL of image showing the map of the location
-    image_url = attr.ib(None, init=False)
-    #: Width of the image
-    image_width = attr.ib(None, init=False)
-    #: Height of the image
-    image_height = attr.ib(None, init=False)
-    #: URL to Bing maps with the location
-    url = attr.ib(None, init=False)
-    # Address of the location
-    address = attr.ib(None)
-
-    # Put here for backwards compatibility, so that the init argument order is preserved
-    uid = attr.ib(None)
-
-
-@attr.s(cmp=False, init=False)
-class LiveLocationAttachment(LocationAttachment):
-    """Represents a live user location"""
-
-    #: Name of the location
-    name = attr.ib(None)
-    #: Timestamp when live location expires
-    expiration_time = attr.ib(None)
-    #: True if live location is expired
-    is_expired = attr.ib(None)
-
-    def __init__(self, name=None, expiration_time=None, is_expired=None, **kwargs):
-        super(LiveLocationAttachment, self).__init__(**kwargs)
-        self.expiration_time = expiration_time
-        self.is_expired = is_expired

fbchat/_message.py

@@ -1,127 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-from string import Formatter
-from ._core import Enum
-
-
-class EmojiSize(Enum):
-    """Used to specify the size of a sent emoji"""
-
-    LARGE = "369239383222810"
-    MEDIUM = "369239343222814"
-    SMALL = "369239263222822"
-
-
-class MessageReaction(Enum):
-    """Used to specify a message reaction"""
-
-    LOVE = "😍"
-    SMILE = "😆"
-    WOW = "😮"
-    SAD = "😢"
-    ANGRY = "😠"
-    YES = "👍"
-    NO = "👎"
-
-
-@attr.s(cmp=False)
-class Mention(object):
-    """Represents a @mention"""
-
-    #: The thread ID the mention is pointing at
-    thread_id = attr.ib()
-    #: The character where the mention starts
-    offset = attr.ib(0)
-    #: The length of the mention
-    length = attr.ib(10)
-
-
-@attr.s(cmp=False)
-class Message(object):
-    """Represents a Facebook message"""
-
-    #: The actual message
-    text = attr.ib(None)
-    #: A list of :class:`Mention` objects
-    mentions = attr.ib(factory=list, converter=lambda x: [] if x is None else x)
-    #: A :class:`EmojiSize`. Size of a sent emoji
-    emoji_size = attr.ib(None)
-    #: The message ID
-    uid = attr.ib(None, init=False)
-    #: ID of the sender
-    author = attr.ib(None, init=False)
-    #: Timestamp of when the message was sent
-    timestamp = attr.ib(None, init=False)
-    #: Whether the message is read
-    is_read = attr.ib(None, init=False)
-    #: A list of pepole IDs who read the message, works only with :func:`fbchat.Client.fetchThreadMessages`
-    read_by = attr.ib(factory=list, init=False)
-    #: A dict with user's IDs as keys, and their :class:`MessageReaction` as values
-    reactions = attr.ib(factory=dict, init=False)
-    #: A :class:`Sticker`
-    sticker = attr.ib(None)
-    #: A list of attachments
-    attachments = attr.ib(factory=list, converter=lambda x: [] if x is None else x)
-    #: A list of :class:`QuickReply`
-    quick_replies = attr.ib(factory=list, converter=lambda x: [] if x is None else x)
-    #: Whether the message is unsent (deleted for everyone)
-    unsent = attr.ib(False, init=False)
-    #: Message ID you want to reply to
-    reply_to_id = attr.ib(None)
-    #: Replied message
-    replied_to = attr.ib(None, init=False)
-
-    @classmethod
-    def formatMentions(cls, text, *args, **kwargs):
-        """Like `str.format`, but takes tuples with a thread id and text instead.
-
-        Returns a `Message` object, with the formatted string and relevant mentions.
-
-        ```
-        >>> Message.formatMentions("Hey {!r}! My name is {}", ("1234", "Peter"), ("4321", "Michael"))
-        <Message (None): "Hey 'Peter'! My name is Michael", mentions=[<Mention 1234: offset=4 length=7>, <Mention 4321: offset=24 length=7>] emoji_size=None attachments=[]>
-
-        >>> Message.formatMentions("Hey {p}! My name is {}", ("1234", "Michael"), p=("4321", "Peter"))
-        <Message (None): 'Hey Peter! My name is Michael', mentions=[<Mention 4321: offset=4 length=5>, <Mention 1234: offset=22 length=7>] emoji_size=None attachments=[]>
-        ```
-        """
-        result = ""
-        mentions = list()
-        offset = 0
-        f = Formatter()
-        field_names = [field_name[1] for field_name in f.parse(text)]
-        automatic = "" in field_names
-        i = 0
-
-        for (literal_text, field_name, format_spec, conversion) in f.parse(text):
-            offset += len(literal_text)
-            result += literal_text
-
-            if field_name is None:
-                continue
-
-            if field_name == "":
-                field_name = str(i)
-                i += 1
-            elif automatic and field_name.isdigit():
-                raise ValueError(
-                    "cannot switch from automatic field numbering to manual field specification"
-                )
-
-            thread_id, name = f.get_field(field_name, args, kwargs)[0]
-
-            if format_spec:
-                name = f.format_field(name, format_spec)
-            if conversion:
-                name = f.convert_field(name, conversion)
-
-            result += name
-            mentions.append(
-                Mention(thread_id=thread_id, offset=offset, length=len(name))
-            )
-            offset += len(name)
-
-        message = cls(text=result, mentions=mentions)
-        return message

fbchat/_models/__init__.py

@@ -0,0 +1,9 @@
+from ._common import *
+from ._attachment import *
+from ._file import *
+from ._location import *
+from ._plan import *
+from ._poll import *
+from ._quick_reply import *
+from ._sticker import *
+from ._message import *

fbchat/_models/_attachment.py

@@ -0,0 +1,81 @@
+import attr
+from . import Image
+from .._common import attrs_default
+from .. import _util
+
+from typing import Optional, Sequence
+
+
+@attrs_default
+class Attachment:
+    """Represents a Facebook attachment."""
+
+    #: The attachment ID
+    id = attr.ib(None, type=Optional[str])
+
+
+@attrs_default
+class UnsentMessage(Attachment):
+    """Represents an unsent message attachment."""
+
+
+@attrs_default
+class ShareAttachment(Attachment):
+    """Represents a shared item (e.g. URL) attachment."""
+
+    #: ID of the author of the shared post
+    author = attr.ib(None, type=Optional[str])
+    #: Target URL
+    url = attr.ib(None, type=Optional[str])
+    #: Original URL if Facebook redirects the URL
+    original_url = attr.ib(None, type=Optional[str])
+    #: Title of the attachment
+    title = attr.ib(None, type=Optional[str])
+    #: Description of the attachment
+    description = attr.ib(None, type=Optional[str])
+    #: Name of the source
+    source = attr.ib(None, type=Optional[str])
+    #: The attached image
+    image = attr.ib(None, type=Optional[Image])
+    #: URL of the original image if Facebook uses ``safe_image``
+    original_image_url = attr.ib(None, type=Optional[str])
+    #: List of additional attachments
+    attachments = attr.ib(factory=list, type=Sequence[Attachment])
+
+    @classmethod
+    def _from_graphql(cls, data):
+        from . import _file
+
+        image = None
+        original_image_url = None
+        media = data.get("media")
+        if media and media.get("image"):
+            image = Image._from_uri(media["image"])
+            original_image_url = (
+                _util.get_url_parameter(image.url, "url")
+                if "/safe_image.php" in image.url
+                else image.url
+            )
+
+        url = data.get("url")
+        return cls(
+            id=data.get("deduplication_key"),
+            author=data["target"]["actors"][0]["id"]
+            if data["target"].get("actors")
+            else None,
+            url=url,
+            original_url=_util.get_url_parameter(url, "u")
+            if "/l.php?u=" in url
+            else url,
+            title=data["title_with_entities"].get("text"),
+            description=data["description"].get("text")
+            if data.get("description")
+            else None,
+            source=data["source"].get("text") if data.get("source") else None,
+            image=image,
+            original_image_url=original_image_url,
+            attachments=[
+                _file.graphql_to_subattachment(attachment)
+                for attachment in data.get("subattachments")
+            ],
+        )

fbchat/_models/_common.py

@@ -0,0 +1,81 @@
+import attr
+import datetime
+import enum
+from .._common import attrs_default
+from .. import _util
+
+from typing import Optional
+
+
+class ThreadLocation(enum.Enum):
+    """Used to specify where a thread is located (inbox, pending, archived, other)."""
+
+    INBOX = "INBOX"
+    PENDING = "PENDING"
+    ARCHIVED = "ARCHIVED"
+    OTHER = "OTHER"
+
+    @classmethod
+    def _parse(cls, value: str):
+        return cls(value.lstrip("FOLDER_"))
+
+
+@attrs_default
+class ActiveStatus:
+    #: Whether the user is active now
+    active = attr.ib(type=bool)
+    #: When the user was last active
+    last_active = attr.ib(None, type=Optional[datetime.datetime])
+    #: Whether the user is playing Messenger game now
+    in_game = attr.ib(None, type=Optional[bool])
+
+    @classmethod
+    def _from_orca_presence(cls, data):
+        # TODO: Handle `c` and `vc` keys (Probably some binary data)
+        return cls(
+            active=data["p"] in [2, 3],
+            last_active=_util.seconds_to_datetime(data["l"]) if "l" in data else None,
+            in_game=None,
+        )
+
+
+@attrs_default
+class Image:
+    #: URL to the image
+    url = attr.ib(type=str)
+    #: Width of the image
+    width = attr.ib(None, type=Optional[int])
+    #: Height of the image
+    height = attr.ib(None, type=Optional[int])
+
+    @classmethod
+    def _from_uri(cls, data):
+        return cls(
+            url=data["uri"],
+            width=int(data["width"]) if data.get("width") else None,
+            height=int(data["height"]) if data.get("height") else None,
+        )
+
+    @classmethod
+    def _from_url(cls, data):
+        return cls(
+            url=data["url"],
+            width=int(data["width"]) if data.get("width") else None,
+            height=int(data["height"]) if data.get("height") else None,
+        )
+
+    @classmethod
+    def _from_uri_or_none(cls, data):
+        if data is None:
+            return None
+        if data.get("uri") is None:
+            return None
+        return cls._from_uri(data)
+
+    @classmethod
+    def _from_url_or_none(cls, data):
+        if data is None:
+            return None
+        if data.get("url") is None:
+            return None
+        return cls._from_url(data)

fbchat/_models/_file.py

@@ -0,0 +1,195 @@
+import attr
+import datetime
+from . import Image, Attachment
+from .._common import attrs_default
+from .. import _util
+
+from typing import Set, Optional
+
+
+@attrs_default
+class FileAttachment(Attachment):
+    """Represents a file that has been sent as a Facebook attachment."""
+
+    #: URL where you can download the file
+    url = attr.ib(None, type=Optional[str])
+    #: Size of the file in bytes
+    size = attr.ib(None, type=Optional[int])
+    #: Name of the file
+    name = attr.ib(None, type=Optional[str])
+    #: Whether Facebook determines that this file may be harmful
+    is_malicious = attr.ib(None, type=Optional[bool])
+
+    @classmethod
+    def _from_graphql(cls, data, size=None):
+        return cls(
+            url=data.get("url"),
+            size=size,
+            name=data.get("filename"),
+            is_malicious=data.get("is_malicious"),
+            id=data.get("message_file_fbid"),
+        )
+
+
+@attrs_default
+class AudioAttachment(Attachment):
+    """Represents an audio file that has been sent as a Facebook attachment."""
+
+    #: Name of the file
+    filename = attr.ib(None, type=Optional[str])
+    #: URL of the audio file
+    url = attr.ib(None, type=Optional[str])
+    #: Duration of the audio clip
+    duration = attr.ib(None, type=Optional[datetime.timedelta])
+    #: Audio type
+    audio_type = attr.ib(None, type=Optional[str])
+
+    @classmethod
+    def _from_graphql(cls, data):
+        return cls(
+            filename=data.get("filename"),
+            url=data.get("playable_url"),
+            duration=_util.millis_to_timedelta(data.get("playable_duration_in_ms")),
+            audio_type=data.get("audio_type"),
+        )
+
+
+@attrs_default
+class ImageAttachment(Attachment):
+    """Represents an image that has been sent as a Facebook attachment.
+
+    To retrieve the full image URL, use: `Client.fetch_image_url`, and pass it the id of
+    the image attachment.
+    """
+
+    #: The extension of the original image (e.g. ``png``)
+    original_extension = attr.ib(None, type=Optional[str])
+    #: Width of original image
+    width = attr.ib(None, converter=_util.int_or_none, type=Optional[int])
+    #: Height of original image
+    height = attr.ib(None, converter=_util.int_or_none, type=Optional[int])
+    #: Whether the image is animated
+    is_animated = attr.ib(None, type=Optional[bool])
+    #: A set, containing variously sized / various types of previews of the image
+    previews = attr.ib(factory=set, type=Set[Image])
+
+    @classmethod
+    def _from_graphql(cls, data):
+        previews = {
+            Image._from_uri_or_none(data.get("thumbnail")),
+            Image._from_uri_or_none(data.get("preview") or data.get("preview_image")),
+            Image._from_uri_or_none(data.get("large_preview")),
+            Image._from_uri_or_none(data.get("animated_image")),
+        }
+
+        return cls(
+            original_extension=data.get("original_extension")
+            or (data["filename"].split("-")[0] if data.get("filename") else None),
+            width=data.get("original_dimensions", {}).get("width"),
+            height=data.get("original_dimensions", {}).get("height"),
+            is_animated=data["__typename"] == "MessageAnimatedImage",
+            previews={p for p in previews if p},
+            id=data.get("legacy_attachment_id"),
+        )
+
+    @classmethod
+    def _from_list(cls, data):
+        previews = {
+            Image._from_uri_or_none(data["image"]),
+            Image._from_uri(data["image1"]),
+            Image._from_uri(data["image2"]),
+        }
+
+        return cls(
+            width=data["original_dimensions"].get("x"),
+            height=data["original_dimensions"].get("y"),
+            previews={p for p in previews if p},
+            id=data["legacy_attachment_id"],
+        )
+
+
+@attrs_default
+class VideoAttachment(Attachment):
+    """Represents a video that has been sent as a Facebook attachment."""
+
+    #: Size of the original video in bytes
+    size = attr.ib(None, type=Optional[int])
+    #: Width of original video
+    width = attr.ib(None, type=Optional[int])
+    #: Height of original video
+    height = attr.ib(None, type=Optional[int])
+    #: Length of video
+    duration = attr.ib(None, type=Optional[datetime.timedelta])
+    #: URL to very compressed preview video
+    preview_url = attr.ib(None, type=Optional[str])
+    #: A set, containing variously sized previews of the video
+    previews = attr.ib(factory=set, type=Set[Image])
+
+    @classmethod
+    def _from_graphql(cls, data, size=None):
+        previews = {
+            Image._from_uri_or_none(data.get("chat_image")),
+            Image._from_uri_or_none(data.get("inbox_image")),
+            Image._from_uri_or_none(data.get("large_image")),
+        }
+
+        return cls(
+            size=size,
+            width=data.get("original_dimensions", {}).get("width"),
+            height=data.get("original_dimensions", {}).get("height"),
+            duration=_util.millis_to_timedelta(data.get("playable_duration_in_ms")),
+            preview_url=data.get("playable_url"),
+            previews={p for p in previews if p},
+            id=data.get("legacy_attachment_id"),
+        )
+
+    @classmethod
+    def _from_subattachment(cls, data):
+        media = data["media"]
+        image = Image._from_uri_or_none(media.get("image"))
+
+        return cls(
+            duration=_util.millis_to_timedelta(media.get("playable_duration_in_ms")),
+            preview_url=media.get("playable_url"),
+            previews={image} if image else {},
+            id=data["target"].get("video_id"),
+        )
+
+    @classmethod
+    def _from_list(cls, data):
+        previews = {
+            Image._from_uri(data["image"]),
+            Image._from_uri(data["image1"]),
+            Image._from_uri(data["image2"]),
+        }
+
+        return cls(
+            width=data["original_dimensions"].get("x"),
+            height=data["original_dimensions"].get("y"),
+            previews=previews,
+            id=data["legacy_attachment_id"],
+        )
+
+
+def graphql_to_attachment(data, size=None):
+    _type = data["__typename"]
+    if _type in ["MessageImage", "MessageAnimatedImage"]:
+        return ImageAttachment._from_graphql(data)
+    elif _type == "MessageVideo":
+        return VideoAttachment._from_graphql(data, size=size)
+    elif _type == "MessageAudio":
+        return AudioAttachment._from_graphql(data)
+    elif _type == "MessageFile":
+        return FileAttachment._from_graphql(data, size=size)
+
+    return Attachment(id=data.get("legacy_attachment_id"))
+
+
+def graphql_to_subattachment(data):
+    target = data.get("target")
+    type_ = target.get("__typename") if target else None
+
+    if type_ == "Video":
+        return VideoAttachment._from_subattachment(data)
+
+    return None

fbchat/_models/_location.py

@@ -0,0 +1,100 @@
+import attr
+import datetime
+from . import Image, Attachment
+from .._common import attrs_default
+from .. import _util, _exception
+
+from typing import Optional
+
+
+@attrs_default
+class LocationAttachment(Attachment):
+    """Represents a user location.
+
+    Latitude and longitude OR address is provided by Facebook.
+    """
+
+    #: Latitude of the location
+    latitude = attr.ib(None, type=Optional[float])
+    #: Longitude of the location
+    longitude = attr.ib(None, type=Optional[float])
+    #: Image showing the map of the location
+    image = attr.ib(None, type=Optional[Image])
+    #: URL to Bing maps with the location
+    url = attr.ib(None, type=Optional[str])
+    # Address of the location
+    address = attr.ib(None, type=Optional[str])
+
+    @classmethod
+    def _from_graphql(cls, data):
+        url = data.get("url")
+        address = _util.get_url_parameter(_util.get_url_parameter(url, "u"), "where1")
+        if not address:
+            raise _exception.ParseError("Could not find location address", data=data)
+        try:
+            latitude, longitude = [float(x) for x in address.split(", ")]
+            address = None
+        except ValueError:
+            latitude, longitude = None, None
+
+        return cls(
+            id=int(data["deduplication_key"]),
+            latitude=latitude,
+            longitude=longitude,
+            image=Image._from_uri_or_none(data["media"].get("image"))
+            if data.get("media")
+            else None,
+            url=url,
+            address=address,
+        )
+
+
+@attrs_default
+class LiveLocationAttachment(LocationAttachment):
+    """Represents a live user location."""
+
+    #: Name of the location
+    name = attr.ib(None, type=Optional[str])
+    #: When live location expires
+    expires_at = attr.ib(None, type=Optional[datetime.datetime])
+    #: True if live location is expired
+    is_expired = attr.ib(None, type=Optional[bool])
+
+    @classmethod
+    def _from_pull(cls, data):
+        return cls(
+            id=data["id"],
+            latitude=data["coordinate"]["latitude"] / (10 ** 8)
+            if not data.get("stopReason")
+            else None,
+            longitude=data["coordinate"]["longitude"] / (10 ** 8)
+            if not data.get("stopReason")
+            else None,
+            name=data.get("locationTitle"),
+            expires_at=_util.millis_to_datetime(data["expirationTime"]),
+            is_expired=bool(data.get("stopReason")),
+        )
+
+    @classmethod
+    def _from_graphql(cls, data):
+        target = data["target"]
+
+        image = None
+        media = data.get("media")
+        if media and media.get("image"):
+            image = Image._from_uri(media["image"])
+
+        return cls(
+            id=int(target["live_location_id"]),
+            latitude=target["coordinate"]["latitude"]
+            if target.get("coordinate")
+            else None,
+            longitude=target["coordinate"]["longitude"]
+            if target.get("coordinate")
+            else None,
+            image=image,
+            url=data.get("url"),
+            name=data["title_with_entities"]["text"],
+            expires_at=_util.seconds_to_datetime(target.get("expiration_time")),
+            is_expired=target.get("is_expired"),
+        )

fbchat/_models/_message.py

@@ -0,0 +1,488 @@
+import attr
+import datetime
+import enum
+from string import Formatter
+from . import _attachment, _location, _file, _quick_reply, _sticker
+from .._common import log, attrs_default
+from .. import _exception, _util
+from typing import Optional, Mapping, Sequence, Any
+
+
+class EmojiSize(enum.Enum):
+    """Used to specify the size of a sent emoji."""
+
+    LARGE = "369239383222810"
+    MEDIUM = "369239343222814"
+    SMALL = "369239263222822"
+
+    @classmethod
+    def _from_tags(cls, tags):
+        string_to_emojisize = {
+            "large": cls.LARGE,
+            "medium": cls.MEDIUM,
+            "small": cls.SMALL,
+            "l": cls.LARGE,
+            "m": cls.MEDIUM,
+            "s": cls.SMALL,
+        }
+        for tag in tags or ():
+            data = tag.split(":", 1)
+            if len(data) > 1 and data[0] == "hot_emoji_size":
+                return string_to_emojisize.get(data[1])
+        return None
+
+
+@attrs_default
+class Mention:
+    """Represents a ``@mention``.
+
+    >>> fbchat.Mention(thread_id="1234", offset=5, length=2)
+    Mention(thread_id="1234", offset=5, length=2)
+    """
+
+    #: The thread ID the mention is pointing at
+    thread_id = attr.ib(type=str)
+    #: The character where the mention starts
+    offset = attr.ib(type=int)
+    #: The length of the mention
+    length = attr.ib(type=int)
+
+    @classmethod
+    def _from_range(cls, data):
+        # TODO: Parse data["entity"]["__typename"]
+        return cls(
+            # Can be missing
+            thread_id=data["entity"].get("id"),
+            offset=data["offset"],
+            length=data["length"],
+        )
+
+    @classmethod
+    def _from_prng(cls, data):
+        return cls(thread_id=data["i"], offset=data["o"], length=data["l"])
+
+    def _to_send_data(self, i):
+        return {
+            "profile_xmd[{}][id]".format(i): self.thread_id,
+            "profile_xmd[{}][offset]".format(i): self.offset,
+            "profile_xmd[{}][length]".format(i): self.length,
+            "profile_xmd[{}][type]".format(i): "p",
+        }
+
+
+# Exaustively searched for options by using the list in:
+# https://unicode.org/emoji/charts/full-emoji-list.html
+SENDABLE_REACTIONS = ("❤", "😍", "😆", "😮", "😢", "😠", "👍", "👎")
+
+
+@attrs_default
+class Message:
+    """Represents a Facebook message.
+
+    Example:
+        >>> thread = fbchat.User(session=session, id="1234")
+        >>> message = fbchat.Message(thread=thread, id="mid.$XYZ")
+    """
+
+    #: The thread that this message belongs to.
+    thread = attr.ib()
+    #: The message ID.
+    id = attr.ib(converter=str, type=str)
+
+    @property
+    def session(self):
+        """The session to use when making requests."""
+        return self.thread.session
+
+    @staticmethod
+    def _delete_many(session, message_ids):
+        data = {}
+        for i, id_ in enumerate(message_ids):
+            data["message_ids[{}]".format(i)] = id_
+        j = session._payload_post("/ajax/mercury/delete_messages.php?dpr=1", data)
+
+    def delete(self):
+        """Delete the message (removes it only for the user).
+
+        If you want to delete multiple messages, please use `Client.delete_messages`.
+
+        Example:
+            >>> message.delete()
+        """
+        self._delete_many(self.session, [self.id])
+
+    def unsend(self):
+        """Unsend the message (removes it for everyone).
+
+        The message must to be sent by you, and less than 10 minutes ago.
+
+        Example:
+            >>> message.unsend()
+        """
+        data = {"message_id": self.id}
+        j = self.session._payload_post("/messaging/unsend_message/?dpr=1", data)
+
+    def react(self, reaction: Optional[str]):
+        """React to the message, or removes reaction.
+
+        Currently, you can use "❤", "😍", "😆", "😮", "😢", "😠", "👍" or "👎". It
+        should be possible to add support for more, but we haven't figured that out yet.
+
+        Args:
+            reaction: Reaction emoji to use, or if ``None``, removes reaction.
+
+        Example:
+            >>> message.react("😍")
+        """
+        if reaction and reaction not in SENDABLE_REACTIONS:
+            raise ValueError(
+                "Invalid reaction! Please use one of: {}".format(SENDABLE_REACTIONS)
+            )
+
+        data = {
+            "action": "ADD_REACTION" if reaction else "REMOVE_REACTION",
+            "client_mutation_id": "1",
+            "actor_id": self.session.user.id,
+            "message_id": self.id,
+            "reaction": reaction,
+        }
+        data = {
+            "doc_id": 1491398900900362,
+            "variables": _util.json_minimal({"data": data}),
+        }
+        j = self.session._payload_post("/webgraphql/mutation", data)
+        _exception.handle_graphql_errors(j)
+
+    def fetch(self) -> "MessageData":
+        """Fetch fresh `MessageData` object.
+
+        Example:
+            >>> message = message.fetch()
+            >>> message.text
+            "The message text"
+        """
+        message_info = self.thread._forced_fetch(self.id).get("message")
+        return MessageData._from_graphql(self.thread, message_info)
+
+    @staticmethod
+    def format_mentions(text, *args, **kwargs):
+        """Like `str.format`, but takes tuples with a thread id and text instead.
+
+        Return a tuple, with the formatted string and relevant mentions.
+
+        >>> Message.format_mentions("Hey {!r}! My name is {}", ("1234", "Peter"), ("4321", "Michael"))
+        ("Hey 'Peter'! My name is Michael", [Mention(thread_id=1234, offset=4, length=7), Mention(thread_id=4321, offset=24, length=7)])
+
+        >>> Message.format_mentions("Hey {p}! My name is {}", ("1234", "Michael"), p=("4321", "Peter"))
+        ('Hey Peter! My name is Michael', [Mention(thread_id=4321, offset=4, length=5), Mention(thread_id=1234, offset=22, length=7)])
+        """
+        result = ""
+        mentions = list()
+        offset = 0
+        f = Formatter()
+        field_names = [field_name[1] for field_name in f.parse(text)]
+        automatic = "" in field_names
+        i = 0
+
+        for (literal_text, field_name, format_spec, conversion) in f.parse(text):
+            offset += len(literal_text)
+            result += literal_text
+
+            if field_name is None:
+                continue
+
+            if field_name == "":
+                field_name = str(i)
+                i += 1
+            elif automatic and field_name.isdigit():
+                raise ValueError(
+                    "cannot switch from automatic field numbering to manual field specification"
+                )
+
+            thread_id, name = f.get_field(field_name, args, kwargs)[0]
+
+            if format_spec:
+                name = f.format_field(name, format_spec)
+            if conversion:
+                name = f.convert_field(name, conversion)
+
+            result += name
+            mentions.append(
+                Mention(thread_id=thread_id, offset=offset, length=len(name))
+            )
+            offset += len(name)
+
+        return result, mentions
+
+
+@attrs_default
+class MessageSnippet(Message):
+    """Represents data in a Facebook message snippet.
+
+    Inherits `Message`.
+    """
+
+    #: ID of the sender
+    author = attr.ib(type=str)
+    #: When the message was sent
+    created_at = attr.ib(type=datetime.datetime)
+    #: The actual message
+    text = attr.ib(type=str)
+    #: A dict with offsets, mapped to the matched text
+    matched_keywords = attr.ib(type=Mapping[int, str])
+
+    @classmethod
+    def _parse(cls, thread, data):
+        return cls(
+            thread=thread,
+            id=data["message_id"],
+            author=data["author"].rstrip("fbid:"),
+            created_at=_util.millis_to_datetime(data["timestamp"]),
+            text=data["body"],
+            matched_keywords={int(k): v for k, v in data["matched_keywords"].items()},
+        )
+
+
+@attrs_default
+class MessageData(Message):
+    """Represents data in a Facebook message.
+
+    Inherits `Message`.
+    """
+
+    #: ID of the sender
+    author = attr.ib(type=str)
+    #: When the message was sent
+    created_at = attr.ib(type=datetime.datetime)
+    #: The actual message
+    text = attr.ib(None, type=Optional[str])
+    #: A list of `Mention` objects
+    mentions = attr.ib(factory=list, type=Sequence[Mention])
+    #: Size of a sent emoji
+    emoji_size = attr.ib(None, type=Optional[EmojiSize])
+    #: Whether the message is read
+    is_read = attr.ib(None, type=Optional[bool])
+    #: People IDs who read the message, only works with `ThreadABC.fetch_messages`
+    read_by = attr.ib(factory=list, type=bool)
+    #: A dictionary with user's IDs as keys, and their reaction as values
+    reactions = attr.ib(factory=dict, type=Mapping[str, str])
+    #: A `Sticker`
+    sticker = attr.ib(None, type=Optional[_sticker.Sticker])
+    #: A list of attachments
+    attachments = attr.ib(factory=list, type=Sequence[_attachment.Attachment])
+    #: A list of `QuickReply`
+    quick_replies = attr.ib(factory=list, type=Sequence[_quick_reply.QuickReply])
+    #: Whether the message is unsent (deleted for everyone)
+    unsent = attr.ib(False, type=Optional[bool])
+    #: Message ID you want to reply to
+    reply_to_id = attr.ib(None, type=Optional[str])
+    #: Replied message
+    replied_to = attr.ib(None, type=Optional[Any])
+    #: Whether the message was forwarded
+    forwarded = attr.ib(False, type=Optional[bool])
+
+    @staticmethod
+    def _get_forwarded_from_tags(tags):
+        if tags is None:
+            return False
+        return any(map(lambda tag: "forward" in tag or "copy" in tag, tags))
+
+    @staticmethod
+    def _parse_quick_replies(data):
+        if data:
+            data = _util.parse_json(data).get("quick_replies")
+            if isinstance(data, list):
+                return [_quick_reply.graphql_to_quick_reply(q) for q in data]
+            elif isinstance(data, dict):
+                return [_quick_reply.graphql_to_quick_reply(data, is_response=True)]
+        return []
+
+    @classmethod
+    def _from_graphql(cls, thread, data, read_receipts=None):
+        if data.get("message_sender") is None:
+            data["message_sender"] = {}
+        if data.get("message") is None:
+            data["message"] = {}
+        tags = data.get("tags_list")
+
+        created_at = _util.millis_to_datetime(int(data.get("timestamp_precise")))
+
+        attachments = [
+            _file.graphql_to_attachment(attachment)
+            for attachment in data.get("blob_attachments") or ()
+        ]
+        unsent = False
+        if data.get("extensible_attachment") is not None:
+            attachment = graphql_to_extensible_attachment(data["extensible_attachment"])
+            if isinstance(attachment, _attachment.UnsentMessage):
+                unsent = True
+            elif attachment:
+                attachments.append(attachment)
+
+        replied_to = None
+        if data.get("replied_to_message") and data["replied_to_message"]["message"]:
+            # data["replied_to_message"]["message"] is None if the message is deleted
+            replied_to = cls._from_graphql(
+                thread, data["replied_to_message"]["message"]
+            )
+
+        return cls(
+            thread=thread,
+            id=str(data["message_id"]),
+            author=str(data["message_sender"]["id"]),
+            created_at=created_at,
+            text=data["message"].get("text"),
+            mentions=[
+                Mention._from_range(m) for m in data["message"].get("ranges") or ()
+            ],
+            emoji_size=EmojiSize._from_tags(tags),
+            is_read=not data["unread"] if data.get("unread") is not None else None,
+            read_by=[
+                receipt["actor"]["id"]
+                for receipt in read_receipts or ()
+                if _util.millis_to_datetime(int(receipt["watermark"])) >= created_at
+            ],
+            reactions={
+                str(r["user"]["id"]): r["reaction"] for r in data["message_reactions"]
+            },
+            sticker=_sticker.Sticker._from_graphql(data.get("sticker")),
+            attachments=attachments,
+            quick_replies=cls._parse_quick_replies(data.get("platform_xmd_encoded")),
+            unsent=unsent,
+            reply_to_id=replied_to.id if replied_to else None,
+            replied_to=replied_to,
+            forwarded=cls._get_forwarded_from_tags(tags),
+        )
+
+    @classmethod
+    def _from_reply(cls, thread, data):
+        tags = data["messageMetadata"].get("tags")
+        metadata = data.get("messageMetadata", {})
+
+        attachments = []
+        unsent = False
+        sticker = None
+        for attachment in data.get("attachments") or ():
+            attachment = _util.parse_json(attachment["mercuryJSON"])
+            if attachment.get("blob_attachment"):
+                attachments.append(
+                    _file.graphql_to_attachment(attachment["blob_attachment"])
+                )
+            if attachment.get("extensible_attachment"):
+                extensible_attachment = graphql_to_extensible_attachment(
+                    attachment["extensible_attachment"]
+                )
+                if isinstance(extensible_attachment, _attachment.UnsentMessage):
+                    unsent = True
+                else:
+                    attachments.append(extensible_attachment)
+            if attachment.get("sticker_attachment"):
+                sticker = _sticker.Sticker._from_graphql(
+                    attachment["sticker_attachment"]
+                )
+
+        return cls(
+            thread=thread,
+            id=metadata.get("messageId"),
+            author=str(metadata["actorFbId"]),
+            created_at=_util.millis_to_datetime(metadata["timestamp"]),
+            text=data.get("body"),
+            mentions=[
+                Mention._from_prng(m)
+                for m in _util.parse_json(data.get("data", {}).get("prng", "[]"))
+            ],
+            emoji_size=EmojiSize._from_tags(tags),
+            sticker=sticker,
+            attachments=attachments,
+            quick_replies=cls._parse_quick_replies(data.get("platform_xmd_encoded")),
+            unsent=unsent,
+            reply_to_id=data["messageReply"]["replyToMessageId"]["id"]
+            if "messageReply" in data
+            else None,
+            forwarded=cls._get_forwarded_from_tags(tags),
+        )
+
+    @classmethod
+    def _from_pull(cls, thread, data, author, created_at):
+        metadata = data["messageMetadata"]
+
+        tags = metadata.get("tags")
+
+        mentions = []
+        if data.get("data") and data["data"].get("prng"):
+            try:
+                mentions = [
+                    Mention._from_prng(m)
+                    for m in _util.parse_json(data["data"]["prng"])
+                ]
+            except Exception:
+                log.exception("An exception occured while reading attachments")
+
+        attachments = []
+        unsent = False
+        sticker = None
+        try:
+            for a in data.get("attachments") or ():
+                mercury = a["mercury"]
+                if mercury.get("blob_attachment"):
+                    image_metadata = a.get("imageMetadata", {})
+                    attach_type = mercury["blob_attachment"]["__typename"]
+                    attachment = _file.graphql_to_attachment(
+                        mercury["blob_attachment"], a.get("fileSize")
+                    )
+                    attachments.append(attachment)
+
+                elif mercury.get("sticker_attachment"):
+                    sticker = _sticker.Sticker._from_graphql(
+                        mercury["sticker_attachment"]
+                    )
+
+                elif mercury.get("extensible_attachment"):
+                    attachment = graphql_to_extensible_attachment(
+                        mercury["extensible_attachment"]
+                    )
+                    if isinstance(attachment, _attachment.UnsentMessage):
+                        unsent = True
+                    elif attachment:
+                        attachments.append(attachment)
+
+        except Exception:
+            log.exception(
+                "An exception occured while reading attachments: {}".format(
+                    data["attachments"]
+                )
+            )
+
+        return cls(
+            thread=thread,
+            id=metadata["messageId"],
+            author=author,
+            created_at=created_at,
+            text=data.get("body"),
+            mentions=mentions,
+            emoji_size=EmojiSize._from_tags(tags),
+            sticker=sticker,
+            attachments=attachments,
+            unsent=unsent,
+            forwarded=cls._get_forwarded_from_tags(tags),
+        )
+
+
+def graphql_to_extensible_attachment(data):
+    story = data.get("story_attachment")
+    if not story:
+        return None
+
+    target = story.get("target")
+    if not target:
+        return _attachment.UnsentMessage(id=data.get("legacy_attachment_id"))
+
+    _type = target["__typename"]
+    if _type == "MessageLocation":
+        return _location.LocationAttachment._from_graphql(story)
+    elif _type == "MessageLiveLocation":
+        return _location.LiveLocationAttachment._from_graphql(story)
+    elif _type in ["ExternalUrl", "Story"]:
+        return _attachment.ShareAttachment._from_graphql(story)
+
+    return None

fbchat/_models/_plan.py

@@ -0,0 +1,212 @@
+import attr
+import datetime
+import enum
+from .._common import attrs_default
+from .. import _exception, _util, _session
+
+from typing import Mapping, Sequence, Optional
+
+
+class GuestStatus(enum.Enum):
+    INVITED = 1
+    GOING = 2
+    DECLINED = 3
+
+
+ACONTEXT = {
+    "action_history": [
+        {"surface": "messenger_chat_tab", "mechanism": "messenger_composer"}
+    ]
+}
+
+
+@attrs_default
+class Plan:
+    """Base model for plans.
+
+    Example:
+        >>> plan = fbchat.Plan(session=session, id="1234")
+    """
+
+    #: The session to use when making requests.
+    session = attr.ib(type=_session.Session)
+    #: The plan's unique identifier.
+    id = attr.ib(converter=str, type=str)
+
+    def fetch(self) -> "PlanData":
+        """Fetch fresh `PlanData` object.
+
+        Example:
+            >>> plan = plan.fetch()
+            >>> plan.title
+            "A plan"
+        """
+        data = {"event_reminder_id": self.id}
+        j = self.session._payload_post("/ajax/eventreminder", data)
+        return PlanData._from_fetch(self.session, j)
+
+    @classmethod
+    def _create(
+        cls,
+        thread,
+        name: str,
+        at: datetime.datetime,
+        location_name: str = None,
+        location_id: str = None,
+    ):
+        data = {
+            "event_type": "EVENT",
+            "event_time": _util.datetime_to_seconds(at),
+            "title": name,
+            "thread_id": thread.id,
+            "location_id": location_id or "",
+            "location_name": location_name or "",
+            "acontext": ACONTEXT,
+        }
+        j = thread.session._payload_post("/ajax/eventreminder/create", data)
+        if "error" in j:
+            raise _exception.ExternalError("Failed creating plan", j["error"])
+
+    def edit(
+        self,
+        name: str,
+        at: datetime.datetime,
+        location_name: str = None,
+        location_id: str = None,
+    ):
+        """Edit the plan.
+
+        # TODO: Arguments
+        """
+        data = {
+            "event_reminder_id": self.id,
+            "delete": "false",
+            "date": _util.datetime_to_seconds(at),
+            "location_name": location_name or "",
+            "location_id": location_id or "",
+            "title": name,
+            "acontext": ACONTEXT,
+        }
+        j = self.session._payload_post("/ajax/eventreminder/submit", data)
+
+    def delete(self):
+        """Delete the plan.
+
+        Example:
+            >>> plan.delete()
+        """
+        data = {"event_reminder_id": self.id, "delete": "true", "acontext": ACONTEXT}
+        j = self.session._payload_post("/ajax/eventreminder/submit", data)
+
+    def _change_participation(self):
+        data = {
+            "event_reminder_id": self.id,
+            "guest_state": "GOING" if take_part else "DECLINED",
+            "acontext": ACONTEXT,
+        }
+        j = self.session._payload_post("/ajax/eventreminder/rsvp", data)
+
+    def participate(self):
+        """Set yourself as GOING/participating to the plan.
+
+        Example:
+            >>> plan.participate()
+        """
+        return self._change_participation(True)
+
+    def decline(self):
+        """Set yourself as having DECLINED the plan.
+
+        Example:
+            >>> plan.decline()
+        """
+        return self._change_participation(False)
+
+
+@attrs_default
+class PlanData(Plan):
+    """Represents data about a plan."""
+
+    #: Plan time, only precise down to the minute
+    time = attr.ib(type=datetime.datetime)
+    #: Plan title
+    title = attr.ib(type=str)
+    #: Plan location name
+    location = attr.ib(None, converter=lambda x: x or "", type=Optional[str])
+    #: Plan location ID
+    location_id = attr.ib(None, converter=lambda x: x or "", type=Optional[str])
+    #: ID of the plan creator
+    author_id = attr.ib(None, type=Optional[str])
+    #: `User` ids mapped to their `GuestStatus`
+    guests = attr.ib(None, type=Optional[Mapping[str, GuestStatus]])
+
+    @property
+    def going(self) -> Sequence[str]:
+        """List of the `User` IDs who will take part in the plan."""
+        return [
+            id_
+            for id_, status in (self.guests or {}).items()
+            if status is GuestStatus.GOING
+        ]
+
+    @property
+    def declined(self) -> Sequence[str]:
+        """List of the `User` IDs who won't take part in the plan."""
+        return [
+            id_
+            for id_, status in (self.guests or {}).items()
+            if status is GuestStatus.DECLINED
+        ]
+
+    @property
+    def invited(self) -> Sequence[str]:
+        """List of the `User` IDs who are invited to the plan."""
+        return [
+            id_
+            for id_, status in (self.guests or {}).items()
+            if status is GuestStatus.INVITED
+        ]
+
+    @classmethod
+    def _from_pull(cls, session, data):
+        return cls(
+            session=session,
+            id=data.get("event_id"),
+            time=_util.seconds_to_datetime(int(data.get("event_time"))),
+            title=data.get("event_title"),
+            location=data.get("event_location_name"),
+            location_id=data.get("event_location_id"),
+            author_id=data.get("event_creator_id"),
+            guests={
+                x["node"]["id"]: GuestStatus[x["guest_list_state"]]
+                for x in _util.parse_json(data["guest_state_list"])
+            },
+        )
+
+    @classmethod
+    def _from_fetch(cls, session, data):
+        return cls(
+            session=session,
+            id=data.get("oid"),
+            time=_util.seconds_to_datetime(data.get("event_time")),
+            title=data.get("title"),
+            location=data.get("location_name"),
+            location_id=str(data["location_id"]) if data.get("location_id") else None,
+            author_id=data.get("creator_id"),
+            guests={id_: GuestStatus[s] for id_, s in data["event_members"].items()},
+        )
+
+    @classmethod
+    def _from_graphql(cls, session, data):
+        return cls(
+            session=session,
+            id=data.get("id"),
+            time=_util.seconds_to_datetime(data.get("time")),
+            title=data.get("event_title"),
+            location=data.get("location_name"),
+            author_id=data["lightweight_event_creator"].get("id"),
+            guests={
+                x["node"]["id"]: GuestStatus[x["guest_list_state"]]
+                for x in data["event_reminder_members"]["edges"]
+            },
+        )

fbchat/_models/_poll.py

@@ -0,0 +1,115 @@
+import attr
+from .._common import attrs_default
+from .. import _exception, _session
+from typing import Iterable, Sequence
+
+
+@attrs_default
+class PollOption:
+    """Represents a poll option."""
+
+    #: ID of the poll option
+    id = attr.ib(converter=str, type=str)
+    #: Text of the poll option
+    text = attr.ib(type=str)
+    #: Whether vote when creating or client voted
+    vote = attr.ib(type=bool)
+    #: ID of the users who voted for this poll option
+    voters = attr.ib(type=Sequence[str])
+    #: Votes count
+    votes_count = attr.ib(type=int)
+
+    @classmethod
+    def _from_graphql(cls, data):
+        if data.get("viewer_has_voted") is None:
+            vote = False
+        elif isinstance(data["viewer_has_voted"], bool):
+            vote = data["viewer_has_voted"]
+        else:
+            vote = data["viewer_has_voted"] == "true"
+        return cls(
+            id=int(data["id"]),
+            text=data.get("text"),
+            vote=vote,
+            voters=(
+                [m["node"]["id"] for m in data["voters"]["edges"]]
+                if isinstance(data.get("voters"), dict)
+                else data["voters"]
+            ),
+            votes_count=(
+                data["voters"]["count"]
+                if isinstance(data.get("voters"), dict)
+                else data["total_count"]
+            ),
+        )
+
+
+@attrs_default
+class Poll:
+    """Represents a poll."""
+
+    #: ID of the poll
+    session = attr.ib(type=_session.Session)
+    #: ID of the poll
+    id = attr.ib(converter=str, type=str)
+    #: The poll's question
+    question = attr.ib(type=str)
+    #: The poll's top few options. The full list can be fetched with `fetch_options`
+    options = attr.ib(type=Sequence[PollOption])
+    #: Options count
+    options_count = attr.ib(type=int)
+
+    @classmethod
+    def _from_graphql(cls, session, data):
+        return cls(
+            session=session,
+            id=data["id"],
+            question=data["title"] if data.get("title") else data["text"],
+            options=[PollOption._from_graphql(m) for m in data["options"]],
+            options_count=data["total_count"],
+        )
+
+    def fetch_options(self) -> Sequence[PollOption]:
+        """Fetch all `PollOption` objects on the poll.
+
+        The result is ordered with options with the most votes first.
+
+        Example:
+            >>> options = poll.fetch_options()
+            >>> options[0].text
+            "An option"
+        """
+        data = {"question_id": self.id}
+        j = self.session._payload_post("/ajax/mercury/get_poll_options", data)
+        return [PollOption._from_graphql(m) for m in j]
+
+    def set_votes(self, option_ids: Iterable[str], new_options: Iterable[str] = None):
+        """Update the user's poll vote.
+
+        Args:
+            option_ids: Option ids to vote for / keep voting for
+            new_options: New options to add
+
+        Example:
+            >>> options = poll.fetch_options()
+            >>> # Add option
+            >>> poll.set_votes([o.id for o in options], new_options=["New option"])
+            >>> # Remove vote from option
+            >>> poll.set_votes([o.id for o in options if o.text != "Option 1"])
+        """
+        data = {"question_id": self.id}
+
+        for i, option_id in enumerate(option_ids or ()):
+            data["selected_options[{}]".format(i)] = option_id
+
+        for i, option_text in enumerate(new_options or ()):
+            data["new_options[{}]".format(i)] = option_text
+
+        j = self.session._payload_post(
+            "/messaging/group_polling/update_vote/?dpr=1", data
+        )
+        if j.get("status") != "success":
+            raise _exception.ExternalError(
+                "Failed updating poll vote: {}".format(j.get("errorTitle")),
+                j.get("errorMessage"),
+            )

fbchat/_models/_quick_reply.py

@@ -0,0 +1,82 @@
+import attr
+from . import Attachment
+from .._common import attrs_default
+
+from typing import Any, Optional
+
+
+@attrs_default
+class QuickReply:
+    """Represents a quick reply."""
+
+    #: Payload of the quick reply
+    payload = attr.ib(None, type=Any)
+    #: External payload for responses
+    external_payload = attr.ib(None, type=Any)
+    #: Additional data
+    data = attr.ib(None, type=Any)
+    #: Whether it's a response for a quick reply
+    is_response = attr.ib(False, type=bool)
+
+
+@attrs_default
+class QuickReplyText(QuickReply):
+    """Represents a text quick reply."""
+
+    #: Title of the quick reply
+    title = attr.ib(None, type=Optional[str])
+    #: URL of the quick reply image
+    image_url = attr.ib(None, type=Optional[str])
+    #: Type of the quick reply
+    _type = "text"
+
+
+@attrs_default
+class QuickReplyLocation(QuickReply):
+    """Represents a location quick reply (Doesn't work on mobile)."""
+
+    #: Type of the quick reply
+    _type = "location"
+
+
+@attrs_default
+class QuickReplyPhoneNumber(QuickReply):
+    """Represents a phone number quick reply (Doesn't work on mobile)."""
+
+    #: URL of the quick reply image
+    image_url = attr.ib(None, type=Optional[str])
+    #: Type of the quick reply
+    _type = "user_phone_number"
+
+
+@attrs_default
+class QuickReplyEmail(QuickReply):
+    """Represents an email quick reply (Doesn't work on mobile)."""
+
+    #: URL of the quick reply image
+    image_url = attr.ib(None, type=Optional[str])
+    #: Type of the quick reply
+    _type = "user_email"
+
+
+def graphql_to_quick_reply(q, is_response=False):
+    data = dict()
+    _type = q.get("content_type").lower()
+    if q.get("payload"):
+        data["payload"] = q["payload"]
+    if q.get("data"):
+        data["data"] = q["data"]
+    if q.get("image_url") and _type is not QuickReplyLocation._type:
+        data["image_url"] = q["image_url"]
+    data["is_response"] = is_response
+    if _type == QuickReplyText._type:
+        if q.get("title") is not None:
+            data["title"] = q["title"]
+        rtn = QuickReplyText(**data)
+    elif _type == QuickReplyLocation._type:
+        rtn = QuickReplyLocation(**data)
+    elif _type == QuickReplyPhoneNumber._type:
+        rtn = QuickReplyPhoneNumber(**data)
+    elif _type == QuickReplyEmail._type:
+        rtn = QuickReplyEmail(**data)
+    return rtn

fbchat/_models/_sticker.py

@@ -0,0 +1,57 @@
+import attr
+from . import Image, Attachment
+from .._common import attrs_default
+
+from typing import Optional
+
+
+@attrs_default
+class Sticker(Attachment):
+    """Represents a Facebook sticker that has been sent to a thread as an attachment."""
+
+    #: The sticker-pack's ID
+    pack = attr.ib(None, type=Optional[str])
+    #: Whether the sticker is animated
+    is_animated = attr.ib(False, type=bool)
+
+    # If the sticker is animated, the following should be present
+    #: URL to a medium spritemap
+    medium_sprite_image = attr.ib(None, type=Optional[str])
+    #: URL to a large spritemap
+    large_sprite_image = attr.ib(None, type=Optional[str])
+    #: The amount of frames present in the spritemap pr. row
+    frames_per_row = attr.ib(None, type=Optional[int])
+    #: The amount of frames present in the spritemap pr. column
+    frames_per_col = attr.ib(None, type=Optional[int])
+    #: The total amount of frames in the spritemap
+    frame_count = attr.ib(None, type=Optional[int])
+    #: The frame rate the spritemap is intended to be played in
+    frame_rate = attr.ib(None, type=Optional[int])
+
+    #: The sticker's image
+    image = attr.ib(None, type=Optional[Image])
+    #: The sticker's label/name
+    label = attr.ib(None, type=Optional[str])
+
+    @classmethod
+    def _from_graphql(cls, data):
+        if not data:
+            return None
+
+        return cls(
+            id=data["id"],
+            pack=data["pack"].get("id") if data.get("pack") else None,
+            is_animated=bool(data.get("sprite_image")),
+            medium_sprite_image=data["sprite_image"].get("uri")
+            if data.get("sprite_image")
+            else None,
+            large_sprite_image=data["sprite_image_2x"].get("uri")
+            if data.get("sprite_image_2x")
+            else None,
+            frames_per_row=data.get("frames_per_row"),
+            frames_per_col=data.get("frames_per_column"),
+            frame_count=data.get("frame_count"),
+            frame_rate=data.get("frame_rate"),
+            image=Image._from_url_or_none(data),
+            label=data["label"] if data.get("label") else None,
+        )

fbchat/_page.py

@@ -1,38 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-from ._thread import ThreadType, Thread
-
-
-@attr.s(cmp=False, init=False)
-class Page(Thread):
-    """Represents a Facebook page. Inherits `Thread`"""
-
-    #: The page's custom url
-    url = attr.ib(None)
-    #: The name of the page's location city
-    city = attr.ib(None)
-    #: Amount of likes the page has
-    likes = attr.ib(None)
-    #: Some extra information about the page
-    sub_title = attr.ib(None)
-    #: The page's category
-    category = attr.ib(None)
-
-    def __init__(
-        self,
-        uid,
-        url=None,
-        city=None,
-        likes=None,
-        sub_title=None,
-        category=None,
-        **kwargs
-    ):
-        super(Page, self).__init__(ThreadType.PAGE, uid, **kwargs)
-        self.url = url
-        self.city = city
-        self.likes = likes
-        self.sub_title = sub_title
-        self.category = category

fbchat/_plan.py

@@ -1,28 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-
-
-@attr.s(cmp=False)
-class Plan(object):
-    """Represents a plan"""
-
-    #: ID of the plan
-    uid = attr.ib(None, init=False)
-    #: Plan time (unix time stamp), only precise down to the minute
-    time = attr.ib(converter=int)
-    #: Plan title
-    title = attr.ib()
-    #: Plan location name
-    location = attr.ib(None, converter=lambda x: x or "")
-    #: Plan location ID
-    location_id = attr.ib(None, converter=lambda x: x or "")
-    #: ID of the plan creator
-    author_id = attr.ib(None, init=False)
-    #: List of the people IDs who will take part in the plan
-    going = attr.ib(factory=list, init=False)
-    #: List of the people IDs who won't take part in the plan
-    declined = attr.ib(factory=list, init=False)
-    #: List of the people IDs who are invited to the plan
-    invited = attr.ib(factory=list, init=False)

fbchat/_poll.py

@@ -1,34 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-
-
-@attr.s(cmp=False)
-class Poll(object):
-    """Represents a poll"""
-
-    #: ID of the poll
-    uid = attr.ib(None, init=False)
-    #: Title of the poll
-    title = attr.ib()
-    #: List of :class:`PollOption`, can be fetched with :func:`fbchat.Client.fetchPollOptions`
-    options = attr.ib()
-    #: Options count
-    options_count = attr.ib(None, init=False)
-
-
-@attr.s(cmp=False)
-class PollOption(object):
-    """Represents a poll option"""
-
-    #: ID of the poll option
-    uid = attr.ib(None, init=False)
-    #: Text of the poll option
-    text = attr.ib()
-    #: Whether vote when creating or client voted
-    vote = attr.ib(False)
-    #: ID of the users who voted for this poll option
-    voters = attr.ib(None, init=False)
-    #: Votes count
-    votes_count = attr.ib(None, init=False)

fbchat/_quick_reply.py

@@ -1,76 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-from ._attachment import Attachment
-
-
-@attr.s(cmp=False)
-class QuickReply(object):
-    """Represents a quick reply"""
-
-    #: Payload of the quick reply
-    payload = attr.ib(None)
-    #: External payload for responses
-    external_payload = attr.ib(None, init=False)
-    #: Additional data
-    data = attr.ib(None)
-    #: Whether it's a response for a quick reply
-    is_response = attr.ib(False)
-
-
-@attr.s(cmp=False, init=False)
-class QuickReplyText(QuickReply):
-    """Represents a text quick reply"""
-
-    #: Title of the quick reply
-    title = attr.ib(None)
-    #: URL of the quick reply image (optional)
-    image_url = attr.ib(None)
-    #: Type of the quick reply
-    _type = "text"
-
-    def __init__(self, title=None, image_url=None, **kwargs):
-        super(QuickReplyText, self).__init__(**kwargs)
-        self.title = title
-        self.image_url = image_url
-
-
-@attr.s(cmp=False, init=False)
-class QuickReplyLocation(QuickReply):
-    """Represents a location quick reply (Doesn't work on mobile)"""
-
-    #: Type of the quick reply
-    _type = "location"
-
-    def __init__(self, **kwargs):
-        super(QuickReplyLocation, self).__init__(**kwargs)
-        self.is_response = False
-
-
-@attr.s(cmp=False, init=False)
-class QuickReplyPhoneNumber(QuickReply):
-    """Represents a phone number quick reply (Doesn't work on mobile)"""
-
-    #: URL of the quick reply image (optional)
-    image_url = attr.ib(None)
-    #: Type of the quick reply
-    _type = "user_phone_number"
-
-    def __init__(self, image_url=None, **kwargs):
-        super(QuickReplyPhoneNumber, self).__init__(**kwargs)
-        self.image_url = image_url
-
-
-@attr.s(cmp=False, init=False)
-class QuickReplyEmail(QuickReply):
-    """Represents an email quick reply (Doesn't work on mobile)"""
-
-    #: URL of the quick reply image (optional)
-    image_url = attr.ib(None)
-    #: Type of the quick reply
-    _type = "user_email"
-
-    def __init__(self, image_url=None, **kwargs):
-        super(QuickReplyEmail, self).__init__(**kwargs)
-        self.image_url = image_url

fbchat/_session.py

@@ -0,0 +1,513 @@
+import attr
+import datetime
+import requests
+import random
+import re
+import json
+
+# TODO: Only import when required
+# Or maybe just replace usage with `html.parser`?
+import bs4
+
+from ._common import log, kw_only
+from . import _graphql, _util, _exception
+
+from typing import Optional, Mapping, Callable, Any
+
+
+SERVER_JS_DEFINE_REGEX = re.compile(
+    r'(?:"ServerJS".{,100}\.handle\({.*"define":)|(?:require\("ServerJSDefine"\)\)?\.handleDefines\()'
+)
+SERVER_JS_DEFINE_JSON_DECODER = json.JSONDecoder()
+
+
+def parse_server_js_define(html: str) -> Mapping[str, Any]:
+    """Parse ``ServerJSDefine`` entries from a HTML document."""
+    # Find points where we should start parsing
+    define_splits = SERVER_JS_DEFINE_REGEX.split(html)
+
+    # TODO: Extract jsmods "require" and "define" from `bigPipe.onPageletArrive`?
+
+    # Skip leading entry
+    _, *define_splits = define_splits
+
+    rtn = []
+    if not define_splits:
+        raise _exception.ParseError("Could not find any ServerJSDefine", data=html)
+    if len(define_splits) < 2:
+        raise _exception.ParseError("Could not find enough ServerJSDefine", data=html)
+    if len(define_splits) > 2:
+        raise _exception.ParseError("Found too many ServerJSDefine", data=define_splits)
+    # Parse entries (should be two)
+    for entry in define_splits:
+        try:
+            parsed, _ = SERVER_JS_DEFINE_JSON_DECODER.raw_decode(entry, idx=0)
+        except json.JSONDecodeError as e:
+            raise _exception.ParseError("Invalid ServerJSDefine", data=entry) from e
+        if not isinstance(parsed, list):
+            raise _exception.ParseError("Invalid ServerJSDefine", data=parsed)
+        rtn.extend(parsed)
+
+    # Convert to a dict
+    return _util.get_jsmods_define(rtn)
+
+
+def base36encode(number: int) -> str:
+    """Convert from Base10 to Base36."""
+    # Taken from https://en.wikipedia.org/wiki/Base36#Python_implementation
+    chars = "0123456789abcdefghijklmnopqrstuvwxyz"
+
+    sign = "-" if number < 0 else ""
+    number = abs(number)
+    result = ""
+
+    while number > 0:
+        number, remainder = divmod(number, 36)
+        result = chars[remainder] + result
+
+    return sign + result
+
+
+def prefix_url(url: str) -> str:
+    if url.startswith("/"):
+        return "https://www.messenger.com" + url
+    return url
+
+
+def generate_message_id(now: datetime.datetime, client_id: str) -> str:
+    k = _util.datetime_to_millis(now)
+    l = int(random.random() * 4294967295)
+    return "<{}:{}-{}@mail.projektitan.com>".format(k, l, client_id)
+
+
+def get_user_id(session: requests.Session) -> str:
+    # TODO: Optimize this `.get_dict()` call!
+    cookies = session.cookies.get_dict()
+    rtn = cookies.get("c_user")
+    if rtn is None:
+        raise _exception.ParseError("Could not find user id", data=cookies)
+    return str(rtn)
+
+
+def session_factory() -> requests.Session:
+    from . import __version__
+
+    session = requests.session()
+    # Override Facebook's locale detection during the login process.
+    # The locale is only used when giving errors back to the user, so giving the errors
+    # back in English makes it easier for users to report.
+    session.cookies = session.cookies = requests.cookies.merge_cookies(
+        session.cookies, {"locale": "en_US"}
+    )
+    session.headers["Referer"] = "https://www.messenger.com/"
+    # We won't try to set a fake user agent to mask our presence!
+    # Facebook allows us access anyhow, and it makes our motives clearer:
+    # We're not trying to cheat Facebook, we simply want to access their service
+    session.headers["User-Agent"] = "fbchat/{}".format(__version__)
+    return session
+
+
+def client_id_factory() -> str:
+    return hex(int(random.random() * 2 ** 31))[2:]
+
+
+def find_form_request(html: str):
+    soup = bs4.BeautifulSoup(html, "html.parser", parse_only=bs4.SoupStrainer("form"))
+
+    form = soup.form
+    if not form:
+        raise _exception.ParseError("Could not find form to submit", data=html)
+
+    url = form.get("action")
+    if not url:
+        raise _exception.ParseError("Could not find url to submit to", data=form)
+
+    # From what I've seen, it'll always do this!
+    if url.startswith("/"):
+        url = "https://www.facebook.com" + url
+
+    # It's okay to set missing values to something crap, the values are localized, and
+    # hence are not available in the raw HTML
+    data = {
+        x["name"]: x.get("value", "[missing]")
+        for x in form.find_all(["input", "button"])
+    }
+    return url, data
+
+
+def two_factor_helper(session: requests.Session, r, on_2fa_callback):
+    url, data = find_form_request(r.content.decode("utf-8"))
+
+    # You don't have to type a code if your device is already saved
+    # Repeats if you get the code wrong
+    while "approvals_code" in data:
+        data["approvals_code"] = on_2fa_callback()
+        log.info("Submitting 2FA code")
+        r = session.post(url, data=data, allow_redirects=False)
+        log.debug("2FA location: %s", r.headers.get("Location"))
+        url, data = find_form_request(r.content.decode("utf-8"))
+
+    # TODO: Can be missing if checkup flow was done on another device in the meantime?
+    if "name_action_selected" in data:
+        data["name_action_selected"] = "save_device"
+        log.info("Saving browser")
+        r = session.post(url, data=data, allow_redirects=False)
+        log.debug("2FA location: %s", r.headers.get("Location"))
+        url = r.headers.get("Location")
+        if url and url.startswith("https://www.messenger.com/login/auth_token/"):
+            return url
+        url, data = find_form_request(r.content.decode("utf-8"))
+
+    log.info("Starting Facebook checkup flow")
+    r = session.post(url, data=data, allow_redirects=False)
+    log.debug("2FA location: %s", r.headers.get("Location"))
+
+    url, data = find_form_request(r.content.decode("utf-8"))
+    if "verification_method" in data:
+        raise _exception.NotLoggedIn(
+            "Your account is locked, and you need to log in using a browser, and verify it there!"
+        )
+    if "submit[This was me]" not in data or "submit[This wasn't me]" not in data:
+        raise _exception.ParseError("Could not fill out form properly (2)", data=data)
+    data["submit[This was me]"] = "[any value]"
+    del data["submit[This wasn't me]"]
+    log.info("Verifying login attempt")
+    r = session.post(url, data=data, allow_redirects=False)
+    log.debug("2FA location: %s", r.headers.get("Location"))
+
+    url, data = find_form_request(r.content.decode("utf-8"))
+    if "name_action_selected" not in data:
+        raise _exception.ParseError("Could not fill out form properly (3)", data=data)
+    data["name_action_selected"] = "save_device"
+    log.info("Saving device again")
+    r = session.post(url, data=data, allow_redirects=False)
+    log.debug("2FA location: %s", r.headers.get("Location"))
+    return r.headers.get("Location")
+
+
+def get_error_data(html: str) -> Optional[str]:
+    """Get error message from a request."""
+    soup = bs4.BeautifulSoup(
+        html, "html.parser", parse_only=bs4.SoupStrainer("form", id="login_form")
+    )
+    # Attempt to extract and format the error string
+    return " ".join(list(soup.stripped_strings)[1:3]) or None
+
+
+def get_fb_dtsg(define) -> Optional[str]:
+    if "DTSGInitData" in define:
+        return define["DTSGInitData"]["token"]
+    elif "DTSGInitialData" in define:
+        return define["DTSGInitialData"]["token"]
+    return None
+
+
+@attr.s(slots=True, kw_only=kw_only, repr=False, eq=False)
+class Session:
+    """Stores and manages state required for most Facebook requests.
+
+    This is the main class, which is used to login to Facebook.
+    """
+
+    _user_id = attr.ib(type=str)
+    _fb_dtsg = attr.ib(type=str)
+    _revision = attr.ib(type=int)
+    _session = attr.ib(factory=session_factory, type=requests.Session)
+    _counter = attr.ib(0, type=int)
+    _client_id = attr.ib(factory=client_id_factory, type=str)
+
+    @property
+    def user(self):
+        """The logged in user."""
+        from . import _threads
+
+        # TODO: Consider caching the result
+
+        return _threads.User(session=self, id=self._user_id)
+
+    def __repr__(self) -> str:
+        # An alternative repr, to illustrate that you can't create the class directly
+        return "<fbchat.Session user_id={}>".format(self._user_id)
+
+    def _get_params(self):
+        self._counter += 1  # TODO: Make this operation atomic / thread-safe
+        return {
+            "__a": 1,
+            "__req": base36encode(self._counter),
+            "__rev": self._revision,
+            "fb_dtsg": self._fb_dtsg,
+        }
+
+    # TODO: Add ability to load previous cookies in here, to avoid 2fa flow
+    @classmethod
+    def login(
+        cls, email: str, password: str, on_2fa_callback: Callable[[], int] = None
+    ):
+        """Login the user, using ``email`` and ``password``.
+
+        Args:
+            email: Facebook ``email``, ``id`` or ``phone number``
+            password: Facebook account password
+            on_2fa_callback: Function that will be called, in case a two factor
+                authentication code is needed. This should return the requested code.
+
+                Tested using SMS and authentication applications. If you have both
+                enabled, you might not receive an SMS code, and you'll have to use the
+                authentication application.
+
+                Note: Facebook limits the amount of codes they will give you, so if you
+                don't receive a code, be patient, and try again later!
+
+        Example:
+            >>> import fbchat
+            >>> import getpass
+            >>> session = fbchat.Session.login(
+            ...     input("Email: "),
+            ...     getpass.getpass(),
+            ...     on_2fa_callback=lambda: input("2FA Code: ")
+            ... )
+            Email: abc@gmail.com
+            Password: ****
+            2FA Code: 123456
+            >>> session.user.id
+            "1234"
+        """
+        session = session_factory()
+
+        data = {
+            # "jazoest": "2754",
+            # "lsd": "AVqqqRUa",
+            "initial_request_id": "x",  # any, just has to be present
+            # "timezone": "-120",
+            # "lgndim": "eyJ3IjoxNDQwLCJoIjo5MDAsImF3IjoxNDQwLCJhaCI6ODc3LCJjIjoyNH0=",
+            # "lgnrnd": "044039_RGm9",
+            "lgnjs": "n",
+            "email": email,
+            "pass": password,
+            "login": "1",
+            "persistent": "1",  # Changes the cookie type to have a long "expires"
+            "default_persistent": "0",
+        }
+
+        try:
+            # Should hit a redirect to https://www.messenger.com/
+            # If this does happen, the session is logged in!
+            r = session.post(
+                "https://www.messenger.com/login/password/",
+                data=data,
+                allow_redirects=False,
+            )
+        except requests.RequestException as e:
+            _exception.handle_requests_error(e)
+        _exception.handle_http_error(r.status_code)
+
+        url = r.headers.get("Location")
+
+        # We weren't redirected, hence the email or password was wrong
+        if not url:
+            error = get_error_data(r.content.decode("utf-8"))
+            raise _exception.NotLoggedIn(error)
+
+        if "checkpoint" in url:
+            if not on_2fa_callback:
+                raise _exception.NotLoggedIn(
+                    "2FA code required! Please supply `on_2fa_callback` to .login"
+                )
+            # Get a facebook.com/checkpoint/start url that handles the 2FA flow
+            # This probably works differently for Messenger-only accounts
+            url = _util.get_url_parameter(url, "next")
+            if not url.startswith("https://www.facebook.com/checkpoint/start/"):
+                raise _exception.ParseError("Failed 2fa flow (1)", data=url)
+
+            r = session.get(url, allow_redirects=False)
+            url = r.headers.get("Location")
+            if not url or not url.startswith("https://www.facebook.com/checkpoint/"):
+                raise _exception.ParseError("Failed 2fa flow (2)", data=url)
+
+            r = session.get(url, allow_redirects=False)
+            url = two_factor_helper(session, r, on_2fa_callback)
+
+            if not url.startswith("https://www.messenger.com/login/auth_token/"):
+                raise _exception.ParseError("Failed 2fa flow (3)", data=url)
+
+            r = session.get(url, allow_redirects=False)
+            url = r.headers.get("Location")
+
+        if url != "https://www.messenger.com/":
+            error = get_error_data(r.content.decode("utf-8"))
+            raise _exception.NotLoggedIn("Failed logging in: {}, {}".format(url, error))
+
+        try:
+            return cls._from_session(session=session)
+        except _exception.NotLoggedIn as e:
+            raise _exception.ParseError("Failed loading session", data=r) from e
+
+    def is_logged_in(self) -> bool:
+        """Send a request to Facebook to check the login status.
+
+        Returns:
+            Whether the user is still logged in
+
+        Example:
+            >>> assert session.is_logged_in()
+        """
+        # Send a request to the login url, to see if we're directed to the home page
+        try:
+            r = self._session.get(prefix_url("/login/"), allow_redirects=False)
+        except requests.RequestException as e:
+            _exception.handle_requests_error(e)
+        _exception.handle_http_error(r.status_code)
+        return "https://www.messenger.com/" == r.headers.get("Location")
+
+    def logout(self) -> None:
+        """Safely log out the user.
+
+        The session object must not be used after this action has been performed!
+
+        Example:
+            >>> session.logout()
+        """
+        data = {"fb_dtsg": self._fb_dtsg}
+        try:
+            r = self._session.post(
+                prefix_url("/logout/"), data=data, allow_redirects=False
+            )
+        except requests.RequestException as e:
+            _exception.handle_requests_error(e)
+        _exception.handle_http_error(r.status_code)
+
+        if "Location" not in r.headers:
+            raise _exception.FacebookError("Failed logging out, was not redirected!")
+        if "https://www.messenger.com/login/" != r.headers["Location"]:
+            raise _exception.FacebookError(
+                "Failed logging out, got bad redirect: {}".format(r.headers["Location"])
+            )
+
+    @classmethod
+    def _from_session(cls, session):
+        # TODO: Automatically set user_id when the cookie changes in the session
+        user_id = get_user_id(session)
+
+        # Make a request to the main page to retrieve ServerJSDefine entries
+        try:
+            r = session.get(prefix_url("/"), allow_redirects=False)
+        except requests.RequestException as e:
+            _exception.handle_requests_error(e)
+        _exception.handle_http_error(r.status_code)
+
+        define = parse_server_js_define(r.content.decode("utf-8"))
+
+        fb_dtsg = get_fb_dtsg(define)
+        if fb_dtsg is None:
+            raise _exception.ParseError("Could not find fb_dtsg", data=define)
+        if not fb_dtsg:
+            # Happens when the client is not actually logged in
+            raise _exception.NotLoggedIn(
+                "Found empty fb_dtsg, the session was probably invalid."
+            )
+
+        try:
+            revision = int(define["SiteData"]["client_revision"])
+        except TypeError:
+            raise _exception.ParseError("Could not find client revision", data=define)
+
+        return cls(user_id=user_id, fb_dtsg=fb_dtsg, revision=revision, session=session)
+
+    def get_cookies(self) -> Mapping[str, str]:
+        """Retrieve session cookies, that can later be used in `from_cookies`.
+
+        Returns:
+            A dictionary containing session cookies
+
+        Example:
+            >>> cookies = session.get_cookies()
+        """
+        return self._session.cookies.get_dict()
+
+    @classmethod
+    def from_cookies(cls, cookies: Mapping[str, str]):
+        """Load a session from session cookies.
+
+        Args:
+            cookies: A dictionary containing session cookies
+
+        Example:
+            >>> cookies = session.get_cookies()
+            >>> # Store cookies somewhere, and then subsequently
+            >>> session = fbchat.Session.from_cookies(cookies)
+        """
+        session = session_factory()
+        session.cookies = requests.cookies.merge_cookies(session.cookies, cookies)
+        return cls._from_session(session=session)
+
+    def _post(self, url, data, files=None, as_graphql=False):
+        data.update(self._get_params())
+        try:
+            r = self._session.post(prefix_url(url), data=data, files=files)
+        except requests.RequestException as e:
+            _exception.handle_requests_error(e)
+        # Facebook's encoding is always UTF-8
+        r.encoding = "utf-8"
+        _exception.handle_http_error(r.status_code)
+        if r.text is None or len(r.text) == 0:
+            raise _exception.HTTPError("Error when sending request: Got empty response")
+        if as_graphql:
+            return _graphql.response_to_json(r.text)
+        else:
+            text = _util.strip_json_cruft(r.text)
+            j = _util.parse_json(text)
+            log.debug(j)
+            return j
+
+    def _payload_post(self, url, data, files=None):
+        j = self._post(url, data, files=files)
+        _exception.handle_payload_error(j)
+
+        # update fb_dtsg token if received in response
+        if "jsmods" in j:
+            define = _util.get_jsmods_define(j["jsmods"]["define"])
+            fb_dtsg = get_fb_dtsg(define)
+            if fb_dtsg:
+                self._fb_dtsg = fb_dtsg
+
+        try:
+            return j["payload"]
+        except (KeyError, TypeError) as e:
+            raise _exception.ParseError("Missing payload", data=j) from e
+
+    def _graphql_requests(self, *queries):
+        # TODO: Explain usage of GraphQL, probably in the docs
+        # Perhaps provide this API as public?
+        data = {
+            "method": "GET",
+            "response_format": "json",
+            "queries": _graphql.queries_to_json(*queries),
+        }
+        return self._post("/api/graphqlbatch/", data, as_graphql=True)
+
+    def _do_send_request(self, data):
+        now = _util.now()
+        offline_threading_id = _util.generate_offline_threading_id()
+        data["client"] = "mercury"
+        data["author"] = "fbid:{}".format(self._user_id)
+        data["timestamp"] = _util.datetime_to_millis(now)
+        data["source"] = "source:chat:web"
+        data["offline_threading_id"] = offline_threading_id
+        data["message_id"] = offline_threading_id
+        data["threading_id"] = generate_message_id(now, self._client_id)
+        data["ephemeral_ttl_mode:"] = "0"
+        j = self._post("/messaging/send/", data)
+
+        _exception.handle_payload_error(j)
+
+        try:
+            message_ids = [
+                (action["message_id"], action["thread_fbid"])
+                for action in j["payload"]["actions"]
+                if "message_id" in action
+            ]
+            if len(message_ids) != 1:
+                log.warning("Got multiple message ids' back: {}".format(message_ids))
+            return message_ids[0]
+        except (KeyError, IndexError, TypeError) as e:
+            raise _exception.ParseError("No message IDs could be found", data=j) from e

fbchat/_sticker.py

@@ -1,39 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-from ._attachment import Attachment
-
-
-@attr.s(cmp=False, init=False)
-class Sticker(Attachment):
-    """Represents a Facebook sticker that has been sent to a thread as an attachment"""
-
-    #: The sticker-pack's ID
-    pack = attr.ib(None)
-    #: Whether the sticker is animated
-    is_animated = attr.ib(False)
-
-    # If the sticker is animated, the following should be present
-    #: URL to a medium spritemap
-    medium_sprite_image = attr.ib(None)
-    #: URL to a large spritemap
-    large_sprite_image = attr.ib(None)
-    #: The amount of frames present in the spritemap pr. row
-    frames_per_row = attr.ib(None)
-    #: The amount of frames present in the spritemap pr. coloumn
-    frames_per_col = attr.ib(None)
-    #: The frame rate the spritemap is intended to be played in
-    frame_rate = attr.ib(None)
-
-    #: URL to the sticker's image
-    url = attr.ib(None)
-    #: Width of the sticker
-    width = attr.ib(None)
-    #: Height of the sticker
-    height = attr.ib(None)
-    #: The sticker's label/name
-    label = attr.ib(None)
-
-    def __init__(self, uid=None):
-        super(Sticker, self).__init__(uid=uid)

fbchat/_thread.py

@@ -1,86 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-from ._core import Enum
-
-
-class ThreadType(Enum):
-    """Used to specify what type of Facebook thread is being used. See :ref:`intro_threads` for more info"""
-
-    USER = 1
-    GROUP = 2
-    ROOM = 2
-    PAGE = 3
-
-
-class ThreadLocation(Enum):
-    """Used to specify where a thread is located (inbox, pending, archived, other)."""
-
-    INBOX = "INBOX"
-    PENDING = "PENDING"
-    ARCHIVED = "ARCHIVED"
-    OTHER = "OTHER"
-
-
-class ThreadColor(Enum):
-    """Used to specify a thread colors"""
-
-    MESSENGER_BLUE = "#0084ff"
-    VIKING = "#44bec7"
-    GOLDEN_POPPY = "#ffc300"
-    RADICAL_RED = "#fa3c4c"
-    SHOCKING = "#d696bb"
-    PICTON_BLUE = "#6699cc"
-    FREE_SPEECH_GREEN = "#13cf13"
-    PUMPKIN = "#ff7e29"
-    LIGHT_CORAL = "#e68585"
-    MEDIUM_SLATE_BLUE = "#7646ff"
-    DEEP_SKY_BLUE = "#20cef5"
-    FERN = "#67b868"
-    CAMEO = "#d4a88c"
-    BRILLIANT_ROSE = "#ff5ca1"
-    BILOBA_FLOWER = "#a695c7"
-    TICKLE_ME_PINK = "#ff7ca8"
-    MALACHITE = "#1adb5b"
-    RUBY = "#f01d6a"
-    DARK_TANGERINE = "#ff9c19"
-    BRIGHT_TURQUOISE = "#0edcde"
-
-
-@attr.s(cmp=False, init=False)
-class Thread(object):
-    """Represents a Facebook thread"""
-
-    #: The unique identifier of the thread. Can be used a `thread_id`. See :ref:`intro_threads` for more info
-    uid = attr.ib(converter=str)
-    #: Specifies the type of thread. Can be used a `thread_type`. See :ref:`intro_threads` for more info
-    type = attr.ib()
-    #: A url to the thread's picture
-    photo = attr.ib(None)
-    #: The name of the thread
-    name = attr.ib(None)
-    #: Timestamp of last message
-    last_message_timestamp = attr.ib(None)
-    #: Number of messages in the thread
-    message_count = attr.ib(None)
-    #: Set :class:`Plan`
-    plan = attr.ib(None)
-
-    def __init__(
-        self,
-        _type,
-        uid,
-        photo=None,
-        name=None,
-        last_message_timestamp=None,
-        message_count=None,
-        plan=None,
-    ):
-        self.uid = str(uid)
-        self.type = _type
-        self.photo = photo
-        self.name = name
-        self.last_message_timestamp = last_message_timestamp
-        self.message_count = message_count
-        self.plan = plan

fbchat/_threads/__init__.py

@@ -0,0 +1,4 @@
+from ._abc import *
+from ._group import *
+from ._user import *
+from ._page import *

fbchat/_threads/_abc.py

@@ -0,0 +1,822 @@
+import abc
+import attr
+import collections
+import datetime
+from .._common import log, attrs_default
+from .. import _util, _exception, _session, _graphql, _models
+from typing import MutableMapping, Mapping, Any, Iterable, Tuple, Optional
+
+
+DEFAULT_COLOR = "#0084ff"
+SETABLE_COLORS = (
+    DEFAULT_COLOR,
+    "#44bec7",
+    "#ffc300",
+    "#fa3c4c",
+    "#d696bb",
+    "#6699cc",
+    "#13cf13",
+    "#ff7e29",
+    "#e68585",
+    "#7646ff",
+    "#20cef5",
+    "#67b868",
+    "#d4a88c",
+    "#ff5ca1",
+    "#a695c7",
+    "#ff7ca8",
+    "#1adb5b",
+    "#f01d6a",
+    "#ff9c19",
+    "#0edcde",
+)
+
+
+class ThreadABC(metaclass=abc.ABCMeta):
+    """Implemented by thread-like classes.
+
+    This is private to implement.
+    """
+
+    @property
+    @abc.abstractmethod
+    def session(self) -> _session.Session:
+        """The session to use when making requests."""
+        raise NotImplementedError
+
+    @property
+    @abc.abstractmethod
+    def id(self) -> str:
+        """The unique identifier of the thread."""
+        raise NotImplementedError
+
+    @abc.abstractmethod
+    def _to_send_data(self) -> MutableMapping[str, str]:
+        raise NotImplementedError
+
+    # Note:
+    # You can go out of Facebook's spec with `self.session._do_send_request`!
+    #
+    # A few examples:
+    # - You can send a sticker and an emoji at the same time
+    # - You can wave, send a sticker and text at the same time
+    # - You can reply to a message with a sticker
+    #
+    # We won't support those use cases, it'll make for a confusing API!
+    # If we absolutely need to in the future, we can always add extra functionality
+
+    @abc.abstractmethod
+    def _copy(self) -> "ThreadABC":
+        """It may or may not be a good idea to attach the current thread to new objects.
+
+        So for now, we use this method to create a new thread.
+
+        This should return the minimal representation of the thread (e.g. not UserData).
+        """
+        raise NotImplementedError
+
+    def fetch(self):
+        # TODO: This
+        raise NotImplementedError
+
+    def wave(self, first: bool = True) -> str:
+        """Wave hello to the thread.
+
+        Args:
+            first: Whether to wave first or wave back
+
+        Example:
+            Wave back to the thread.
+
+            >>> thread.wave(False)
+        """
+        data = self._to_send_data()
+        data["action_type"] = "ma-type:user-generated-message"
+        data["lightweight_action_attachment[lwa_state]"] = (
+            "INITIATED" if first else "RECIPROCATED"
+        )
+        data["lightweight_action_attachment[lwa_type]"] = "WAVE"
+        message_id, thread_id = self.session._do_send_request(data)
+        return message_id
+
+    def send_text(
+        self,
+        text: str,
+        mentions: Iterable["_models.Mention"] = None,
+        files: Iterable[Tuple[str, str]] = None,
+        reply_to_id: str = None,
+    ) -> str:
+        """Send a message to the thread.
+
+        Args:
+            text: Text to send
+            mentions: Optional mentions
+            files: Optional tuples, each containing an uploaded file's ID and mimetype.
+                See `ThreadABC.send_files` for an example.
+            reply_to_id: Optional message to reply to
+
+        Example:
+            Send a message with a mention to a thread.
+
+            >>> mention = fbchat.Mention(thread_id="1234", offset=5, length=2)
+            >>> message_id = thread.send_text("A message", mentions=[mention])
+
+            Reply to the message.
+
+            >>> thread.send_text("A reply", reply_to_id=message_id)
+
+        Returns:
+            The sent message
+        """
+        data = self._to_send_data()
+        data["action_type"] = "ma-type:user-generated-message"
+        if text is not None:  # To support `send_files`
+            data["body"] = text
+
+        for i, mention in enumerate(mentions or ()):
+            data.update(mention._to_send_data(i))
+
+        if files:
+            data["has_attachment"] = True
+
+        for i, (file_id, mimetype) in enumerate(files or ()):
+            data["{}s[{}]".format(_util.mimetype_to_key(mimetype), i)] = file_id
+
+        if reply_to_id:
+            data["replied_to_message_id"] = reply_to_id
+
+        return self.session._do_send_request(data)
+
+    def send_emoji(self, emoji: str, size: "_models.EmojiSize") -> str:
+        """Send an emoji to the thread.
+
+        Args:
+            emoji: The emoji to send
+            size: The size of the emoji
+
+        Example:
+            >>> thread.send_emoji("😀", size=fbchat.EmojiSize.LARGE)
+
+        Returns:
+            The sent message
+        """
+        data = self._to_send_data()
+        data["action_type"] = "ma-type:user-generated-message"
+        data["body"] = emoji
+        data["tags[0]"] = "hot_emoji_size:{}".format(size.name.lower())
+        return self.session._do_send_request(data)
+
+    def send_sticker(self, sticker_id: str) -> str:
+        """Send a sticker to the thread.
+
+        Args:
+            sticker_id: ID of the sticker to send
+
+        Example:
+            Send a sticker with the id "1889713947839631"
+
+            >>> thread.send_sticker("1889713947839631")
+
+        Returns:
+            The sent message
+        """
+        data = self._to_send_data()
+        data["action_type"] = "ma-type:user-generated-message"
+        data["sticker_id"] = sticker_id
+        return self.session._do_send_request(data)
+
+    def _send_location(self, current, latitude, longitude):
+        data = self._to_send_data()
+        data["action_type"] = "ma-type:user-generated-message"
+        data["location_attachment[coordinates][latitude]"] = latitude
+        data["location_attachment[coordinates][longitude]"] = longitude
+        data["location_attachment[is_current_location]"] = current
+        return self.session._do_send_request(data)
+
+    def send_location(self, latitude: float, longitude: float):
+        """Send a given location to a thread as the user's current location.
+
+        Args:
+            latitude: The location latitude
+            longitude: The location longitude
+
+        Example:
+            Send a location in London, United Kingdom.
+
+            >>> thread.send_location(51.5287718, -0.2416815)
+        """
+        self._send_location(True, latitude=latitude, longitude=longitude)
+
+    def send_pinned_location(self, latitude: float, longitude: float):
+        """Send a given location to a thread as a pinned location.
+
+        Args:
+            latitude: The location latitude
+            longitude: The location longitude
+
+        Example:
+            Send a pinned location in Beijing, China.
+
+            >>> thread.send_pinned_location(39.9390731, 116.117273)
+        """
+        self._send_location(False, latitude=latitude, longitude=longitude)
+
+    def send_files(self, files: Iterable[Tuple[str, str]]):
+        """Send files from file IDs to a thread.
+
+        `files` should be a list of tuples, with a file's ID and mimetype.
+
+        Example:
+            Upload and send a video to a thread.
+
+            >>> with open("video.mp4", "rb") as f:
+            ...     files = client.upload([("video.mp4", f, "video/mp4")])
+            >>>
+            >>> thread.send_files(files)
+        """
+        return self.send_text(text=None, files=files)
+
+    # xmd = {"quick_replies": []}
+    # for quick_reply in quick_replies:
+    #     # TODO: Move this to `_quick_reply.py`
+    #     q = dict()
+    #     q["content_type"] = quick_reply._type
+    #     q["payload"] = quick_reply.payload
+    #     q["external_payload"] = quick_reply.external_payload
+    #     q["data"] = quick_reply.data
+    #     if quick_reply.is_response:
+    #         q["ignore_for_webhook"] = False
+    #     if isinstance(quick_reply, _quick_reply.QuickReplyText):
+    #         q["title"] = quick_reply.title
+    #     if not isinstance(quick_reply, _quick_reply.QuickReplyLocation):
+    #         q["image_url"] = quick_reply.image_url
+    #     xmd["quick_replies"].append(q)
+    # if len(quick_replies) == 1 and quick_replies[0].is_response:
+    #     xmd["quick_replies"] = xmd["quick_replies"][0]
+    # data["platform_xmd"] = _util.json_minimal(xmd)
+
+    # TODO: This!
+    # def quick_reply(self, quick_reply: QuickReply, payload=None):
+    #     """Reply to chosen quick reply.
+    #
+    #     Args:
+    #         quick_reply: Quick reply to reply to
+    #         payload: Optional answer to the quick reply
+    #     """
+    #     if isinstance(quick_reply, QuickReplyText):
+    #         new = QuickReplyText(
+    #             payload=quick_reply.payload,
+    #             external_payload=quick_reply.external_payload,
+    #             data=quick_reply.data,
+    #             is_response=True,
+    #             title=quick_reply.title,
+    #             image_url=quick_reply.image_url,
+    #         )
+    #         return self.send(Message(text=quick_reply.title, quick_replies=[new]))
+    #     elif isinstance(quick_reply, QuickReplyLocation):
+    #         if not isinstance(payload, LocationAttachment):
+    #             raise TypeError("Payload must be an instance of `LocationAttachment`")
+    #         return self.send_location(payload)
+    #     elif isinstance(quick_reply, QuickReplyEmail):
+    #         new = QuickReplyEmail(
+    #             payload=payload if payload else self.get_emails()[0],
+    #             external_payload=quick_reply.payload,
+    #             data=quick_reply.data,
+    #             is_response=True,
+    #             image_url=quick_reply.image_url,
+    #         )
+    #         return self.send(Message(text=payload, quick_replies=[new]))
+    #     elif isinstance(quick_reply, QuickReplyPhoneNumber):
+    #         new = QuickReplyPhoneNumber(
+    #             payload=payload if payload else self.get_phone_numbers()[0],
+    #             external_payload=quick_reply.payload,
+    #             data=quick_reply.data,
+    #             is_response=True,
+    #             image_url=quick_reply.image_url,
+    #         )
+    #         return self.send(Message(text=payload, quick_replies=[new]))
+
+    def _search_messages(self, query, offset, limit):
+        data = {
+            "query": query,
+            "snippetOffset": offset,
+            "snippetLimit": limit,
+            "identifier": "thread_fbid",
+            "thread_fbid": self.id,
+        }
+        j = self.session._payload_post("/ajax/mercury/search_snippets.php?dpr=1", data)
+
+        result = j["search_snippets"][query].get(self.id)
+        if not result:
+            return (0, [])
+
+        thread = self._copy()
+        snippets = [
+            _models.MessageSnippet._parse(thread, snippet)
+            for snippet in result["snippets"]
+        ]
+        return (result["num_total_snippets"], snippets)
+
+    def search_messages(
+        self, query: str, limit: int
+    ) -> Iterable[_models.MessageSnippet]:
+        """Find and get message IDs by query.
+
+        Warning! If someone send a message to the thread that matches the query, while
+        we're searching, some snippets will get returned twice.
+
+        This is fundamentally not fixable, it's just how the endpoint is implemented.
+
+        The returned message snippets are ordered by last sent first.
+
+        Args:
+            query: Text to search for
+            limit: Max. number of message snippets to retrieve
+
+        Example:
+            Fetch the latest message in the thread that matches the query.
+
+            >>> (message,) = thread.search_messages("abc", limit=1)
+            >>> message.text
+            "Some text and abc"
+        """
+        offset = 0
+        # The max limit is measured empirically to 420, safe default chosen below
+        for limit in _util.get_limits(limit, max_limit=50):
+            _, snippets = self._search_messages(query, offset, limit)
+            yield from snippets
+            if len(snippets) < limit:
+                return  # No more data to fetch
+            offset += limit
+
+    def _fetch_messages(self, limit, before):
+        params = {
+            "id": self.id,
+            "message_limit": limit,
+            "load_messages": True,
+            "load_read_receipts": True,
+            # "load_delivery_receipts": False,
+            # "is_work_teamwork_not_putting_muted_in_unreads": False,
+            "before": _util.datetime_to_millis(before) if before else None,
+        }
+        (j,) = self.session._graphql_requests(
+            _graphql.from_doc_id("1860982147341344", params)  # 2696825200377124
+        )
+
+        if j.get("message_thread") is None:
+            raise _exception.ParseError("Could not fetch messages", data=j)
+
+        # TODO: Should we parse the returned thread data, too?
+
+        read_receipts = j["message_thread"]["read_receipts"]["nodes"]
+
+        thread = self._copy()
+        return [
+            _models.MessageData._from_graphql(thread, message, read_receipts)
+            for message in j["message_thread"]["messages"]["nodes"]
+        ]
+
+    def fetch_messages(self, limit: Optional[int]) -> Iterable["_models.Message"]:
+        """Fetch messages in a thread.
+
+        The returned messages are ordered by last sent first.
+
+        Args:
+            limit: Max. number of threads to retrieve. If ``None``, all threads will be
+                retrieved.
+
+        Example:
+            >>> for message in thread.fetch_messages(limit=5)
+            ...     print(message.text)
+            ...
+            A message
+            Another message
+            None
+            A fourth message
+        """
+        # This is measured empirically as 210 in extreme cases, fairly safe default
+        # chosen below
+        MAX_BATCH_LIMIT = 100
+
+        before = None
+        for limit in _util.get_limits(limit, MAX_BATCH_LIMIT):
+            messages = self._fetch_messages(limit, before)
+            messages.reverse()
+
+            if before:
+                # Strip the first messages
+                yield from messages[1:]
+            else:
+                yield from messages
+
+            if len(messages) < MAX_BATCH_LIMIT:
+                return  # No more data to fetch
+
+            before = messages[-1].created_at
+
+    def _fetch_images(self, limit, after):
+        data = {"id": self.id, "first": limit, "after": after}
+        (j,) = self.session._graphql_requests(
+            _graphql.from_query_id("515216185516880", data)
+        )
+
+        if not j[self.id]:
+            raise _exception.ParseError("Could not find images", data=j)
+
+        result = j[self.id]["message_shared_media"]
+
+        rtn = []
+        for edge in result["edges"]:
+            node = edge["node"]
+            type_ = node["__typename"]
+            if type_ == "MessageImage":
+                rtn.append(_models.ImageAttachment._from_list(node))
+            elif type_ == "MessageVideo":
+                rtn.append(_models.VideoAttachment._from_list(node))
+            else:
+                log.warning("Unknown image type %s, data: %s", type_, edge)
+                rtn.append(None)
+
+        # result["page_info"]["has_next_page"] is not correct when limit > 12
+        return (result["page_info"]["end_cursor"], rtn)
+
+    def fetch_images(self, limit: Optional[int]) -> Iterable["_models.Attachment"]:
+        """Fetch images/videos posted in the thread.
+
+        The returned images are ordered by last sent first.
+
+        Args:
+            limit: Max. number of images to retrieve. If ``None``, all images will be
+                retrieved.
+
+        Example:
+            >>> for image in thread.fetch_messages(limit=3)
+            ...     print(image.id)
+            ...
+            1234
+            2345
+        """
+        cursor = None
+        # The max limit on this request is unknown, so we set it reasonably high
+        # This way `limit=None` also still works
+        for limit in _util.get_limits(limit, max_limit=1000):
+            cursor, images = self._fetch_images(limit, cursor)
+            if not images:
+                return  # No more data to fetch
+            for image in images:
+                if image:
+                    yield image
+
+    def set_nickname(self, user_id: str, nickname: str):
+        """Change the nickname of a user in the thread.
+
+        Args:
+            user_id: User that will have their nickname changed
+            nickname: New nickname
+
+        Example:
+            >>> thread.set_nickname("1234", "A nickname")
+        """
+        data = {
+            "nickname": nickname,
+            "participant_id": user_id,
+            "thread_or_other_fbid": self.id,
+        }
+        j = self.session._payload_post(
+            "/messaging/save_thread_nickname/?source=thread_settings&dpr=1", data
+        )
+
+    def set_color(self, color: str):
+        """Change thread color.
+
+        The new color must be one of the following::
+
+            "#0084ff", "#44bec7", "#ffc300", "#fa3c4c", "#d696bb", "#6699cc",
+            "#13cf13", "#ff7e29", "#e68585", "#7646ff", "#20cef5", "#67b868",
+            "#d4a88c", "#ff5ca1", "#a695c7", "#ff7ca8", "#1adb5b", "#f01d6a",
+            "#ff9c19" or "#0edcde".
+
+        This list is subject to change in the future!
+
+        The default when creating a new thread is ``"#0084ff"``.
+
+        Args:
+            color: New thread color
+
+        Example:
+            Set the thread color to "Coral Pink".
+
+            >>> thread.set_color("#e68585")
+        """
+        if color not in SETABLE_COLORS:
+            raise ValueError(
+                "Invalid color! Please use one of: {}".format(SETABLE_COLORS)
+            )
+
+        # Set color to "" if DEFAULT_COLOR. Just how the endpoint works...
+        if color == DEFAULT_COLOR:
+            color = ""
+
+        data = {"color_choice": color, "thread_or_other_fbid": self.id}
+        j = self.session._payload_post(
+            "/messaging/save_thread_color/?source=thread_settings&dpr=1", data
+        )
+
+    # def set_theme(self, theme_id: str):
+    #     data = {
+    #         "client_mutation_id": "0",
+    #         "actor_id": self.session.user.id,
+    #         "thread_id": self.id,
+    #         "theme_id": theme_id,
+    #         "source": "SETTINGS",
+    #     }
+    #     j = self.session._graphql_requests(
+    #         _graphql.from_doc_id("1768656253222505", {"data": data})
+    #     )
+
+    def set_emoji(self, emoji: Optional[str]):
+        """Change thread emoji.
+
+        Args:
+            emoji: New thread emoji. If ``None``, will be set to the default "LIKE" icon
+
+        Example:
+            Set the thread emoji to "😊".
+
+            >>> thread.set_emoji("😊")
+        """
+        data = {"emoji_choice": emoji, "thread_or_other_fbid": self.id}
+        # While changing the emoji, the Facebook web client actually sends multiple
+        # different requests, though only this one is required to make the change.
+        j = self.session._payload_post(
+            "/messaging/save_thread_emoji/?source=thread_settings&dpr=1", data
+        )
+
+    def forward_attachment(self, attachment_id: str):
+        """Forward an attachment.
+
+        Args:
+            attachment_id: Attachment ID to forward
+
+        Example:
+            >>> thread.forward_attachment("1234")
+        """
+        data = {
+            "attachment_id": attachment_id,
+            "recipient_map[{}]".format(_util.generate_offline_threading_id()): self.id,
+        }
+        j = self.session._payload_post("/mercury/attachments/forward/", data)
+        if not j.get("success"):
+            raise _exception.ExternalError("Failed forwarding attachment", j["error"])
+
+    def _set_typing(self, typing):
+        data = {
+            "typ": "1" if typing else "0",
+            "thread": self.id,
+            # TODO: This
+            # "to": self.id if isinstance(self, _user.User) else "",
+            "source": "mercury-chat",
+        }
+        j = self.session._payload_post("/ajax/messaging/typ.php", data)
+
+    def start_typing(self):
+        """Set the current user to start typing in the thread.
+
+        Example:
+            >>> thread.start_typing()
+        """
+        self._set_typing(True)
+
+    def stop_typing(self):
+        """Set the current user to stop typing in the thread.
+
+        Example:
+            >>> thread.stop_typing()
+        """
+        self._set_typing(False)
+
+    def create_plan(
+        self,
+        name: str,
+        at: datetime.datetime,
+        location_name: str = None,
+        location_id: str = None,
+    ):
+        """Create a new plan.
+
+        # TODO: Arguments
+
+        Args:
+            name: Name of the new plan
+            at: When the plan is for
+
+        Example:
+            >>> thread.create_plan(...)
+        """
+        return _models.Plan._create(self, name, at, location_name, location_id)
+
+    def create_poll(self, question: str, options: Mapping[str, bool]):
+        """Create poll in a thread.
+
+        Args:
+            question: The question
+            options: Options and whether you want to select the option
+
+        Example:
+            >>> thread.create_poll("Test poll", {"Option 1": True, "Option 2": False})
+        """
+        # We're using ordered dictionaries, because the Facebook endpoint that parses
+        # the POST parameters is badly implemented, and deals with ordering the options
+        # wrongly. If you can find a way to fix this for the endpoint, or if you find
+        # another endpoint, please do suggest it ;)
+        data = collections.OrderedDict(
+            [("question_text", question), ("target_id", self.id)]
+        )
+
+        for i, (text, vote) in enumerate(options.items()):
+            data["option_text_array[{}]".format(i)] = text
+            data["option_is_selected_array[{}]".format(i)] = "1" if vote else "0"
+
+        j = self.session._payload_post(
+            "/messaging/group_polling/create_poll/?dpr=1", data
+        )
+        if j.get("status") != "success":
+            raise _exception.ExternalError(
+                "Failed creating poll: {}".format(j.get("errorTitle")),
+                j.get("errorMessage"),
+            )
+
+    def mute(self, duration: datetime.timedelta = None):
+        """Mute the thread.
+
+        Args:
+            duration: Time to mute, use ``None`` to mute forever
+
+        Example:
+            >>> import datetime
+            >>> thread.mute(datetime.timedelta(days=2))
+        """
+        if duration is None:
+            setting = "-1"
+        else:
+            setting = str(_util.timedelta_to_seconds(duration))
+        data = {"mute_settings": setting, "thread_fbid": self.id}
+        j = self.session._payload_post(
+            "/ajax/mercury/change_mute_thread.php?dpr=1", data
+        )
+
+    def unmute(self):
+        """Unmute the thread.
+
+        Example:
+            >>> thread.unmute()
+        """
+        return self.mute(datetime.timedelta(0))
+
+    def _mute_reactions(self, mode: bool):
+        data = {"reactions_mute_mode": "1" if mode else "0", "thread_fbid": self.id}
+        j = self.session._payload_post(
+            "/ajax/mercury/change_reactions_mute_thread/?dpr=1", data
+        )
+
+    def mute_reactions(self):
+        """Mute thread reactions."""
+        self._mute_reactions(True)
+
+    def unmute_reactions(self):
+        """Unmute thread reactions."""
+        self._mute_reactions(False)
+
+    def _mute_mentions(self, mode: bool):
+        data = {"mentions_mute_mode": "1" if mode else "0", "thread_fbid": self.id}
+        j = self.session._payload_post(
+            "/ajax/mercury/change_mentions_mute_thread/?dpr=1", data
+        )
+
+    def mute_mentions(self):
+        """Mute thread mentions."""
+        self._mute_mentions(True)
+
+    def unmute_mentions(self):
+        """Unmute thread mentions."""
+        self._mute_mentions(False)
+
+    def mark_as_spam(self):
+        """Mark the thread as spam, and delete it."""
+        data = {"id": self.id}
+        j = self.session._payload_post("/ajax/mercury/mark_spam.php?dpr=1", data)
+
+    @staticmethod
+    def _delete_many(session, thread_ids):
+        data = {}
+        for i, id_ in enumerate(thread_ids):
+            data["ids[{}]".format(i)] = id_
+        # Not needed any more
+        # j = session._payload_post("/ajax/mercury/change_pinned_status.php?dpr=1", ...)
+        # Both /ajax/mercury/delete_threads.php (with an s) doesn't work
+        j = session._payload_post("/ajax/mercury/delete_thread.php", data)
+
+    def delete(self):
+        """Delete the thread.
+
+        If you want to delete multiple threads, please use `Client.delete_threads`.
+
+        Example:
+            >>> message.delete()
+        """
+        self._delete_many(self.session, [self.id])
+
+    def _forced_fetch(self, message_id: str) -> dict:
+        params = {
+            "thread_and_message_id": {"thread_id": self.id, "message_id": message_id}
+        }
+        (j,) = self.session._graphql_requests(
+            _graphql.from_doc_id("1768656253222505", params)
+        )
+        return j
+
+    @staticmethod
+    def _parse_color(inp: Optional[str]) -> str:
+        if not inp:
+            return DEFAULT_COLOR
+        # Strip the alpha value, and lower the string
+        return "#{}".format(inp[2:].lower())
+
+    @staticmethod
+    def _parse_customization_info(data: Any) -> MutableMapping[str, Any]:
+        if not data or not data.get("customization_info"):
+            return {"emoji": None, "color": DEFAULT_COLOR}
+        info = data["customization_info"]
+
+        rtn = {
+            "emoji": info.get("emoji"),
+            "color": ThreadABC._parse_color(info.get("outgoing_bubble_color")),
+        }
+        if (
+            data.get("thread_type") == "GROUP"
+            or data.get("is_group_thread")
+            or data.get("thread_key", {}).get("thread_fbid")
+        ):
+            rtn["nicknames"] = {}
+            for k in info.get("participant_customizations", []):
+                rtn["nicknames"][k["participant_id"]] = k.get("nickname")
+        elif info.get("participant_customizations"):
+            user_id = data.get("thread_key", {}).get("other_user_id") or data.get("id")
+            pc = info["participant_customizations"]
+            if len(pc) > 0:
+                if pc[0].get("participant_id") == user_id:
+                    rtn["nickname"] = pc[0].get("nickname")
+                else:
+                    rtn["own_nickname"] = pc[0].get("nickname")
+            if len(pc) > 1:
+                if pc[1].get("participant_id") == user_id:
+                    rtn["nickname"] = pc[1].get("nickname")
+                else:
+                    rtn["own_nickname"] = pc[1].get("nickname")
+        return rtn
+
+    @staticmethod
+    def _parse_participants(session, data) -> Iterable["ThreadABC"]:
+        from . import _user, _group, _page
+
+        for node in data["nodes"]:
+            actor = node["messaging_actor"]
+            typename = actor["__typename"]
+            thread_id = actor["id"]
+            if typename == "User":
+                yield _user.User(session=session, id=thread_id)
+            elif typename == "MessageThread":
+                # MessageThread => Group thread
+                yield _group.Group(session=session, id=thread_id)
+            elif typename == "Page":
+                yield _page.Page(session=session, id=thread_id)
+            elif typename == "Group":
+                # We don't handle Facebook "Groups"
+                pass
+            else:
+                log.warning("Unknown type %r in %s", typename, data)
+
+
+@attrs_default
+class Thread(ThreadABC):
+    """Represents a Facebook thread, where the actual type is unknown.
+
+    Implements parts of `ThreadABC`, call the method to figure out if your use case is
+    supported. Otherwise, you'll have to use an `User`/`Group`/`Page` object.
+
+    Note: This list may change in minor versions!
+    """
+
+    #: The session to use when making requests.
+    session = attr.ib(type=_session.Session)
+    #: The unique identifier of the thread.
+    id = attr.ib(converter=str, type=str)
+
+    def _to_send_data(self):
+        raise NotImplementedError(
+            "The method you called is not supported on raw Thread objects."
+            " Please use an appropriate User/Group/Page object instead!"
+        )
+
+    def _copy(self) -> "Thread":
+        return Thread(session=self.session, id=self.id)

fbchat/_threads/_group.py

@@ -0,0 +1,279 @@
+import attr
+import datetime
+from ._abc import ThreadABC
+from . import _user
+from .._common import attrs_default
+from .. import _util, _session, _graphql, _models
+
+from typing import Sequence, Iterable, Set, Mapping, Optional
+
+
+@attrs_default
+class Group(ThreadABC):
+    """Represents a Facebook group. Implements `ThreadABC`.
+
+    Example:
+        >>> group = fbchat.Group(session=session, id="1234")
+    """
+
+    #: The session to use when making requests.
+    session = attr.ib(type=_session.Session)
+    #: The group's unique identifier.
+    id = attr.ib(converter=str, type=str)
+
+    def _to_send_data(self):
+        return {"thread_fbid": self.id}
+
+    def _copy(self) -> "Group":
+        return Group(session=self.session, id=self.id)
+
+    def add_participants(self, user_ids: Iterable[str]):
+        """Add users to the group.
+
+        Args:
+            user_ids: One or more user IDs to add
+
+        Example:
+            >>> group.add_participants(["1234", "2345"])
+        """
+        data = self._to_send_data()
+
+        data["action_type"] = "ma-type:log-message"
+        data["log_message_type"] = "log:subscribe"
+
+        for i, user_id in enumerate(user_ids):
+            if user_id == self.session.user.id:
+                raise ValueError(
+                    "Error when adding users: Cannot add self to group thread"
+                )
+            else:
+                data[
+                    "log_message_data[added_participants][{}]".format(i)
+                ] = "fbid:{}".format(user_id)
+
+        return self.session._do_send_request(data)
+
+    def remove_participant(self, user_id: str):
+        """Remove user from the group.
+
+        Args:
+            user_id: User ID to remove
+
+        Example:
+            >>> group.remove_participant("1234")
+        """
+        data = {"uid": user_id, "tid": self.id}
+        j = self.session._payload_post("/chat/remove_participants/", data)
+
+    def _admin_status(self, user_ids: Iterable[str], status: bool):
+        data = {"add": status, "thread_fbid": self.id}
+
+        for i, user_id in enumerate(user_ids):
+            data["admin_ids[{}]".format(i)] = str(user_id)
+
+        j = self.session._payload_post("/messaging/save_admins/?dpr=1", data)
+
+    def add_admins(self, user_ids: Iterable[str]):
+        """Set specified users as group admins.
+
+        Args:
+            user_ids: One or more user IDs to set admin
+
+        Example:
+            >>> group.add_admins(["1234", "2345"])
+        """
+        self._admin_status(user_ids, True)
+
+    def remove_admins(self, user_ids: Iterable[str]):
+        """Remove admin status from specified users.
+
+        Args:
+            user_ids: One or more user IDs to remove admin
+
+        Example:
+            >>> group.remove_admins(["1234", "2345"])
+        """
+        self._admin_status(user_ids, False)
+
+    def set_title(self, title: str):
+        """Change title of the group.
+
+        Args:
+            title: New title
+
+        Example:
+            >>> group.set_title("Abc")
+        """
+        data = {"thread_name": title, "thread_id": self.id}
+        j = self.session._payload_post("/messaging/set_thread_name/?dpr=1", data)
+
+    def set_image(self, image_id: str):
+        """Change the group image from an image id.
+
+        Args:
+            image_id: ID of uploaded image
+
+        Example:
+            Upload an image, and use it as the group image.
+
+            >>> with open("image.png", "rb") as f:
+            ...     (file,) = client.upload([("image.png", f, "image/png")])
+            ...
+            >>> group.set_image(file[0])
+        """
+        data = {"thread_image_id": image_id, "thread_id": self.id}
+        j = self.session._payload_post("/messaging/set_thread_image/?dpr=1", data)
+
+    def set_approval_mode(self, require_admin_approval: bool):
+        """Change the group's approval mode.
+
+        Args:
+            require_admin_approval: True or False
+
+        Example:
+            >>> group.set_approval_mode(False)
+        """
+        data = {"set_mode": int(require_admin_approval), "thread_fbid": self.id}
+        j = self.session._payload_post("/messaging/set_approval_mode/?dpr=1", data)
+
+    def _users_approval(self, user_ids: Iterable[str], approve: bool):
+        data = {
+            "client_mutation_id": "0",
+            "actor_id": self.session.user.id,
+            "thread_fbid": self.id,
+            "user_ids": list(user_ids),
+            "response": "ACCEPT" if approve else "DENY",
+            "surface": "ADMIN_MODEL_APPROVAL_CENTER",
+        }
+        (j,) = self.session._graphql_requests(
+            _graphql.from_doc_id("1574519202665847", {"data": data})
+        )
+
+    def accept_users(self, user_ids: Iterable[str]):
+        """Accept users to the group from the group's approval.
+
+        Args:
+            user_ids: One or more user IDs to accept
+
+        Example:
+            >>> group.accept_users(["1234", "2345"])
+        """
+        self._users_approval(user_ids, True)
+
+    def deny_users(self, user_ids: Iterable[str]):
+        """Deny users from joining the group.
+
+        Args:
+            user_ids: One or more user IDs to deny
+
+        Example:
+            >>> group.deny_users(["1234", "2345"])
+        """
+        self._users_approval(user_ids, False)
+
+
+@attrs_default
+class GroupData(Group):
+    """Represents data about a Facebook group.
+
+    Inherits `Group`, and implements `ThreadABC`.
+    """
+
+    #: The group's picture
+    photo = attr.ib(None, type=Optional[_models.Image])
+    #: The name of the group
+    name = attr.ib(None, type=Optional[str])
+    #: When the group was last active / when the last message was sent
+    last_active = attr.ib(None, type=Optional[datetime.datetime])
+    #: Number of messages in the group
+    message_count = attr.ib(None, type=Optional[int])
+    #: Set `Plan`
+    plan = attr.ib(None, type=Optional[_models.PlanData])
+    #: The group thread's participant user ids
+    participants = attr.ib(factory=set, type=Set[str])
+    #: A dictionary, containing user nicknames mapped to their IDs
+    nicknames = attr.ib(factory=dict, type=Mapping[str, str])
+    #: The groups's message color
+    color = attr.ib(None, type=Optional[str])
+    #: The groups's default emoji
+    emoji = attr.ib(None, type=Optional[str])
+    # User ids of thread admins
+    admins = attr.ib(factory=set, type=Set[str])
+    # True if users need approval to join
+    approval_mode = attr.ib(None, type=Optional[bool])
+    # Set containing user IDs requesting to join
+    approval_requests = attr.ib(factory=set, type=Set[str])
+    # Link for joining group
+    join_link = attr.ib(None, type=Optional[str])
+
+    @classmethod
+    def _from_graphql(cls, session, data):
+        if data.get("image") is None:
+            data["image"] = {}
+        c_info = cls._parse_customization_info(data)
+        last_active = None
+        if "last_message" in data:
+            last_active = _util.millis_to_datetime(
+                int(data["last_message"]["nodes"][0]["timestamp_precise"])
+            )
+        plan = None
+        if data.get("event_reminders") and data["event_reminders"].get("nodes"):
+            plan = _models.PlanData._from_graphql(
+                session, data["event_reminders"]["nodes"][0]
+            )
+
+        return cls(
+            session=session,
+            id=data["thread_key"]["thread_fbid"],
+            participants=list(
+                cls._parse_participants(session, data["all_participants"])
+            ),
+            nicknames=c_info.get("nicknames"),
+            color=c_info["color"],
+            emoji=c_info["emoji"],
+            admins=set([node.get("id") for node in data.get("thread_admins")]),
+            approval_mode=bool(data.get("approval_mode"))
+            if data.get("approval_mode") is not None
+            else None,
+            approval_requests=set(
+                node["requester"]["id"]
+                for node in data["group_approval_queue"]["nodes"]
+            )
+            if data.get("group_approval_queue")
+            else None,
+            join_link=data["joinable_mode"].get("link"),
+            photo=_models.Image._from_uri_or_none(data["image"]),
+            name=data.get("name"),
+            message_count=data.get("messages_count"),
+            last_active=last_active,
+            plan=plan,
+        )
+
+
+@attrs_default
+class NewGroup(ThreadABC):
+    """Helper class to create new groups.
+
+    TODO: Complete this!
+
+    Construct this class with the desired users, and call a method like `wave`, to...
+    """
+
+    #: The session to use when making requests.
+    session = attr.ib(type=_session.Session)
+    #: The users that should be added to the group.
+    _users = attr.ib(type=Sequence["_user.User"])
+
+    @property
+    def id(self):
+        raise NotImplementedError(
+            "The method you called is not supported on NewGroup objects."
+            " Please use the supported methods to create the group, before attempting"
+            " to call the method."
+        )
+
+    def _to_send_data(self) -> dict:
+        return {
+            "specific_to_list[{}]".format(i): "fbid:{}".format(user.id)
+            for i, user in enumerate(self._users)
+        }

fbchat/_threads/_page.py

@@ -0,0 +1,82 @@
+import attr
+import datetime
+from ._abc import ThreadABC
+from .._common import attrs_default
+from .. import _session, _models
+
+from typing import Optional
+
+
+@attrs_default
+class Page(ThreadABC):
+    """Represents a Facebook page. Implements `ThreadABC`.
+
+    Example:
+        >>> page = fbchat.Page(session=session, id="1234")
+    """
+
+    # TODO: Implement pages properly, the implementation is lacking in a lot of places!
+
+    #: The session to use when making requests.
+    session = attr.ib(type=_session.Session)
+    #: The unique identifier of the page.
+    id = attr.ib(converter=str, type=str)
+
+    def _to_send_data(self):
+        return {"other_user_fbid": self.id}
+
+    def _copy(self) -> "Page":
+        return Page(session=self.session, id=self.id)
+
+
+@attrs_default
+class PageData(Page):
+    """Represents data about a Facebook page.
+
+    Inherits `Page`, and implements `ThreadABC`.
+    """
+
+    #: The page's picture
+    photo = attr.ib(type=_models.Image)
+    #: The name of the page
+    name = attr.ib(type=str)
+    #: When the thread was last active / when the last message was sent
+    last_active = attr.ib(None, type=Optional[datetime.datetime])
+    #: Number of messages in the thread
+    message_count = attr.ib(None, type=Optional[int])
+    #: Set `Plan`
+    plan = attr.ib(None, type=Optional[_models.PlanData])
+    #: The page's custom URL
+    url = attr.ib(None, type=Optional[str])
+    #: The name of the page's location city
+    city = attr.ib(None, type=Optional[str])
+    #: Amount of likes the page has
+    likes = attr.ib(None, type=Optional[int])
+    #: Some extra information about the page
+    sub_title = attr.ib(None, type=Optional[str])
+    #: The page's category
+    category = attr.ib(None, type=Optional[str])
+
+    @classmethod
+    def _from_graphql(cls, session, data):
+        if data.get("profile_picture") is None:
+            data["profile_picture"] = {}
+        if data.get("city") is None:
+            data["city"] = {}
+        plan = None
+        if data.get("event_reminders") and data["event_reminders"].get("nodes"):
+            plan = _models.PlanData._from_graphql(
+                session, data["event_reminders"]["nodes"][0]
+            )
+
+        return cls(
+            session=session,
+            id=data["id"],
+            url=data.get("url"),
+            city=data.get("city").get("name"),
+            category=data.get("category_type"),
+            photo=_models.Image._from_uri(data["profile_picture"]),
+            name=data["name"],
+            message_count=data.get("messages_count"),
+            plan=plan,
+        )

fbchat/_threads/_user.py

@@ -0,0 +1,221 @@
+import attr
+import datetime
+from ._abc import ThreadABC
+from .._common import log, attrs_default
+from .. import _util, _session, _models
+
+from typing import Optional
+
+
+GENDERS = {
+    # For standard requests
+    0: "unknown",
+    1: "female_singular",
+    2: "male_singular",
+    3: "female_singular_guess",
+    4: "male_singular_guess",
+    5: "mixed",
+    6: "neuter_singular",
+    7: "unknown_singular",
+    8: "female_plural",
+    9: "male_plural",
+    10: "neuter_plural",
+    11: "unknown_plural",
+    # For graphql requests
+    "UNKNOWN": "unknown",
+    "FEMALE": "female_singular",
+    "MALE": "male_singular",
+    # '': 'female_singular_guess',
+    # '': 'male_singular_guess',
+    # '': 'mixed',
+    "NEUTER": "neuter_singular",
+    # '': 'unknown_singular',
+    # '': 'female_plural',
+    # '': 'male_plural',
+    # '': 'neuter_plural',
+    # '': 'unknown_plural',
+}
+
+
+@attrs_default
+class User(ThreadABC):
+    """Represents a Facebook user. Implements `ThreadABC`.
+
+    Example:
+        >>> user = fbchat.User(session=session, id="1234")
+    """
+
+    #: The session to use when making requests.
+    session = attr.ib(type=_session.Session)
+    #: The user's unique identifier.
+    id = attr.ib(converter=str, type=str)
+
+    def _to_send_data(self):
+        return {
+            "other_user_fbid": self.id,
+            # The entry below is to support .wave
+            "specific_to_list[0]": "fbid:{}".format(self.id),
+        }
+
+    def _copy(self) -> "User":
+        return User(session=self.session, id=self.id)
+
+    def confirm_friend_request(self):
+        """Confirm a friend request, adding the user to your friend list.
+
+        Example:
+            >>> user.confirm_friend_request()
+        """
+        data = {"to_friend": self.id, "action": "confirm"}
+        j = self.session._payload_post("/ajax/add_friend/action.php?dpr=1", data)
+
+    def remove_friend(self):
+        """Remove the user from the client's friend list.
+
+        Example:
+            >>> user.remove_friend()
+        """
+        data = {"uid": self.id}
+        j = self.session._payload_post("/ajax/profile/removefriendconfirm.php", data)
+
+    def block(self):
+        """Block messages from the user.
+
+        Example:
+            >>> user.block()
+        """
+        data = {"fbid": self.id}
+        j = self.session._payload_post("/messaging/block_messages/?dpr=1", data)
+
+    def unblock(self):
+        """Unblock a previously blocked user.
+
+        Example:
+            >>> user.unblock()
+        """
+        data = {"fbid": self.id}
+        j = self.session._payload_post("/messaging/unblock_messages/?dpr=1", data)
+
+
+@attrs_default
+class UserData(User):
+    """Represents data about a Facebook user.
+
+    Inherits `User`, and implements `ThreadABC`.
+    """
+
+    #: The user's picture
+    photo = attr.ib(type=_models.Image)
+    #: The name of the user
+    name = attr.ib(type=str)
+    #: Whether the user and the client are friends
+    is_friend = attr.ib(type=bool)
+    #: The users first name
+    first_name = attr.ib(type=str)
+    #: The users last name
+    last_name = attr.ib(None, type=Optional[str])
+    #: When the thread was last active / when the last message was sent
+    last_active = attr.ib(None, type=Optional[datetime.datetime])
+    #: Number of messages in the thread
+    message_count = attr.ib(None, type=Optional[int])
+    #: Set `Plan`
+    plan = attr.ib(None, type=Optional[_models.PlanData])
+    #: The profile URL. ``None`` for Messenger-only users
+    url = attr.ib(None, type=Optional[str])
+    #: The user's gender
+    gender = attr.ib(None, type=Optional[str])
+    #: From 0 to 1. How close the client is to the user
+    affinity = attr.ib(None, type=Optional[float])
+    #: The user's nickname
+    nickname = attr.ib(None, type=Optional[str])
+    #: The clients nickname, as seen by the user
+    own_nickname = attr.ib(None, type=Optional[str])
+    #: The message color
+    color = attr.ib(None, type=Optional[str])
+    #: The default emoji
+    emoji = attr.ib(None, type=Optional[str])
+
+    @staticmethod
+    def _get_other_user(data):
+        (user,) = (
+            node["messaging_actor"]
+            for node in data["all_participants"]["nodes"]
+            if node["messaging_actor"]["id"] == data["thread_key"]["other_user_id"]
+        )
+        return user
+
+    @classmethod
+    def _from_graphql(cls, session, data):
+        c_info = cls._parse_customization_info(data)
+
+        plan = None
+        if data.get("event_reminders") and data["event_reminders"].get("nodes"):
+            plan = _models.PlanData._from_graphql(
+                session, data["event_reminders"]["nodes"][0]
+            )
+
+        return cls(
+            session=session,
+            id=data["id"],
+            url=data["url"],
+            first_name=data["first_name"],
+            last_name=data.get("last_name"),
+            is_friend=data["is_viewer_friend"],
+            gender=GENDERS.get(data["gender"]),
+            affinity=data.get("viewer_affinity"),
+            nickname=c_info.get("nickname"),
+            color=c_info["color"],
+            emoji=c_info["emoji"],
+            own_nickname=c_info.get("own_nickname"),
+            photo=_models.Image._from_uri(data["profile_picture"]),
+            name=data["name"],
+            message_count=data.get("messages_count"),
+            plan=plan,
+        )
+
+    @classmethod
+    def _from_thread_fetch(cls, session, data):
+        user = cls._get_other_user(data)
+        if user["__typename"] != "User":
+            # TODO: Add Page._from_thread_fetch, and parse it there
+            log.warning("Tried to parse %s as a user.", user["__typename"])
+            return None
+
+        c_info = cls._parse_customization_info(data)
+
+        plan = None
+        if data["event_reminders"]["nodes"]:
+            plan = _models.PlanData._from_graphql(
+                session, data["event_reminders"]["nodes"][0]
+            )
+
+        return cls(
+            session=session,
+            id=user["id"],
+            url=user["url"],
+            name=user["name"],
+            first_name=user["short_name"],
+            is_friend=user["is_viewer_friend"],
+            gender=GENDERS.get(user["gender"]),
+            nickname=c_info.get("nickname"),
+            color=c_info["color"],
+            emoji=c_info["emoji"],
+            own_nickname=c_info.get("own_nickname"),
+            photo=_models.Image._from_uri(user["big_image_src"]),
+            message_count=data["messages_count"],
+            last_active=_util.millis_to_datetime(int(data["updated_time_precise"])),
+            plan=plan,
+        )
+
+    @classmethod
+    def _from_all_fetch(cls, session, data):
+        return cls(
+            session=session,
+            id=data["id"],
+            first_name=data["firstName"],
+            url=data["uri"],
+            photo=_models.Image(url=data["thumbSrc"]),
+            name=data["name"],
+            is_friend=data["is_friend"],
+            gender=GENDERS.get(data["gender"]),
+        )

fbchat/_user.py

@@ -1,76 +0,0 @@
-# -*- coding: UTF-8 -*-
-from __future__ import unicode_literals
-
-import attr
-from ._core import Enum
-from ._thread import ThreadType, Thread
-
-
-class TypingStatus(Enum):
-    """Used to specify whether the user is typing or has stopped typing"""
-
-    STOPPED = 0
-    TYPING = 1
-
-
-@attr.s(cmp=False, init=False)
-class User(Thread):
-    """Represents a Facebook user. Inherits `Thread`"""
-
-    #: The profile url
-    url = attr.ib(None)
-    #: The users first name
-    first_name = attr.ib(None)
-    #: The users last name
-    last_name = attr.ib(None)
-    #: Whether the user and the client are friends
-    is_friend = attr.ib(None)
-    #: The user's gender
-    gender = attr.ib(None)
-    #: From 0 to 1. How close the client is to the user
-    affinity = attr.ib(None)
-    #: The user's nickname
-    nickname = attr.ib(None)
-    #: The clients nickname, as seen by the user
-    own_nickname = attr.ib(None)
-    #: A :class:`ThreadColor`. The message color
-    color = attr.ib(None)
-    #: The default emoji
-    emoji = attr.ib(None)
-
-    def __init__(
-        self,
-        uid,
-        url=None,
-        first_name=None,
-        last_name=None,
-        is_friend=None,
-        gender=None,
-        affinity=None,
-        nickname=None,
-        own_nickname=None,
-        color=None,
-        emoji=None,
-        **kwargs
-    ):
-        super(User, self).__init__(ThreadType.USER, uid, **kwargs)
-        self.url = url
-        self.first_name = first_name
-        self.last_name = last_name
-        self.is_friend = is_friend
-        self.gender = gender
-        self.affinity = affinity
-        self.nickname = nickname
-        self.own_nickname = own_nickname
-        self.color = color
-        self.emoji = emoji
-
-
-@attr.s(cmp=False)
-class ActiveStatus(object):
-    #: Whether the user is active now
-    active = attr.ib(None)
-    #: Timestamp when the user was last active
-    last_active = attr.ib(None)
-    #: Whether the user is playing Messenger game now
-    in_game = attr.ib(None)

fbchat/_util.py

@@ -1,322 +1,94 @@
-# -*- coding: UTF-8 -*-
+import datetime
-
-from __future__ import unicode_literals
-import re
 import json
-from time import time
+import time
-from random import random
+import random
-from contextlib import contextmanager
+import urllib.parse
-from mimetypes import guess_type
-from os.path import basename
-import warnings
-import logging
-import requests
-import aenum
-from .models import *
 
+from ._common import log
+from . import _exception
+
+from typing import Iterable, Optional, Any, Mapping, Sequence
+
+
+def int_or_none(inp: Any) -> Optional[int]:
     try:
-    from urllib.parse import urlencode, parse_qs, urlparse
+        return int(inp)
-
+    except Exception:
-    basestring = (str, bytes)
+        return None
-except ImportError:
-    from urllib import urlencode
-    from urlparse import parse_qs, urlparse
-
-    basestring = basestring
-
-# Python 2's `input` executes the input, whereas `raw_input` just returns the input
-try:
-    input = raw_input
-except NameError:
-    pass
-
-# Log settings
-log = logging.getLogger("client")
-log.setLevel(logging.DEBUG)
-# Creates the console handler
-handler = logging.StreamHandler()
-log.addHandler(handler)
-
-#: Default list of user agents
-USER_AGENTS = [
-    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36",
-    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/601.1.10 (KHTML, like Gecko) Version/8.0.5 Safari/601.1.10",
-    "Mozilla/5.0 (Windows NT 6.3; WOW64; ; NCT50_AAP285C84A1328) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36",
-    "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/22.0.1207.1 Safari/537.1",
-    "Mozilla/5.0 (X11; CrOS i686 2268.111.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11",
-    "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.0 Safari/536.6",
-]
-
-LIKES = {
-    "large": EmojiSize.LARGE,
-    "medium": EmojiSize.MEDIUM,
-    "small": EmojiSize.SMALL,
-    "l": EmojiSize.LARGE,
-    "m": EmojiSize.MEDIUM,
-    "s": EmojiSize.SMALL,
-}
 
 
-GENDERS = {
+def get_limits(limit: Optional[int], max_limit: int) -> Iterable[int]:
-    # For standard requests
+    """Helper that generates limits based on a max limit."""
-    0: "unknown",
+    if limit is None:
-    1: "female_singular",
+        # Generate infinite items
-    2: "male_singular",
+        while True:
-    3: "female_singular_guess",
+            yield max_limit
-    4: "male_singular_guess",
+
-    5: "mixed",
+    if limit < 0:
-    6: "neuter_singular",
+        raise ValueError("Limit cannot be negative")
-    7: "unknown_singular",
+
-    8: "female_plural",
+    # Generate n items
-    9: "male_plural",
+    yield from [max_limit] * (limit // max_limit)
-    10: "neuter_plural",
+
-    11: "unknown_plural",
+    remainder = limit % max_limit
-    # For graphql requests
+    if remainder:
-    "UNKNOWN": "unknown",
+        yield remainder
-    "FEMALE": "female_singular",
-    "MALE": "male_singular",
-    # '': 'female_singular_guess',
-    # '': 'male_singular_guess',
-    # '': 'mixed',
-    "NEUTER": "neuter_singular",
-    # '': 'unknown_singular',
-    # '': 'female_plural',
-    # '': 'male_plural',
-    # '': 'neuter_plural',
-    # '': 'unknown_plural',
-}
 
 
-class ReqUrl(object):
+def json_minimal(data: Any) -> str:
-    """A class containing all urls used by `fbchat`"""
+    """Get JSON data in minimal form."""
-
+    return json.dumps(data, separators=(",", ":"))
-    SEARCH = "https://www.facebook.com/ajax/typeahead/search.php"
-    LOGIN = "https://m.facebook.com/login.php?login_attempt=1"
-    SEND = "https://www.facebook.com/messaging/send/"
-    UNREAD_THREADS = "https://www.facebook.com/ajax/mercury/unread_threads.php"
-    UNSEEN_THREADS = "https://www.facebook.com/mercury/unseen_thread_ids/"
-    THREADS = "https://www.facebook.com/ajax/mercury/threadlist_info.php"
-    MOVE_THREAD = "https://www.facebook.com/ajax/mercury/move_thread.php"
-    ARCHIVED_STATUS = (
-        "https://www.facebook.com/ajax/mercury/change_archived_status.php?dpr=1"
-    )
-    PINNED_STATUS = (
-        "https://www.facebook.com/ajax/mercury/change_pinned_status.php?dpr=1"
-    )
-    MESSAGES = "https://www.facebook.com/ajax/mercury/thread_info.php"
-    READ_STATUS = "https://www.facebook.com/ajax/mercury/change_read_status.php"
-    DELIVERED = "https://www.facebook.com/ajax/mercury/delivery_receipts.php"
-    MARK_SEEN = "https://www.facebook.com/ajax/mercury/mark_seen.php"
-    BASE = "https://www.facebook.com"
-    MOBILE = "https://m.facebook.com/"
-    STICKY = "https://0-edge-chat.facebook.com/pull"
-    PING = "https://0-edge-chat.facebook.com/active_ping"
-    UPLOAD = "https://upload.facebook.com/ajax/mercury/upload.php"
-    INFO = "https://www.facebook.com/chat/user_info/"
-    CONNECT = "https://www.facebook.com/ajax/add_friend/action.php?dpr=1"
-    REMOVE_USER = "https://www.facebook.com/chat/remove_participants/"
-    LOGOUT = "https://www.facebook.com/logout.php"
-    ALL_USERS = "https://www.facebook.com/chat/user_info_all"
-    SAVE_DEVICE = "https://m.facebook.com/login/save-device/cancel/"
-    CHECKPOINT = "https://m.facebook.com/login/checkpoint/"
-    THREAD_COLOR = "https://www.facebook.com/messaging/save_thread_color/?source=thread_settings&dpr=1"
-    THREAD_NICKNAME = "https://www.facebook.com/messaging/save_thread_nickname/?source=thread_settings&dpr=1"
-    THREAD_EMOJI = "https://www.facebook.com/messaging/save_thread_emoji/?source=thread_settings&dpr=1"
-    THREAD_IMAGE = "https://www.facebook.com/messaging/set_thread_image/?dpr=1"
-    THREAD_NAME = "https://www.facebook.com/messaging/set_thread_name/?dpr=1"
-    MESSAGE_REACTION = "https://www.facebook.com/webgraphql/mutation"
-    TYPING = "https://www.facebook.com/ajax/messaging/typ.php"
-    GRAPHQL = "https://www.facebook.com/api/graphqlbatch/"
-    ATTACHMENT_PHOTO = "https://www.facebook.com/mercury/attachments/photo/"
-    PLAN_CREATE = "https://www.facebook.com/ajax/eventreminder/create"
-    PLAN_INFO = "https://www.facebook.com/ajax/eventreminder"
-    PLAN_CHANGE = "https://www.facebook.com/ajax/eventreminder/submit"
-    PLAN_PARTICIPATION = "https://www.facebook.com/ajax/eventreminder/rsvp"
-    MODERN_SETTINGS_MENU = "https://www.facebook.com/bluebar/modern_settings_menu/"
-    REMOVE_FRIEND = "https://m.facebook.com/a/removefriend.php"
-    BLOCK_USER = "https://www.facebook.com/messaging/block_messages/?dpr=1"
-    UNBLOCK_USER = "https://www.facebook.com/messaging/unblock_messages/?dpr=1"
-    SAVE_ADMINS = "https://www.facebook.com/messaging/save_admins/?dpr=1"
-    APPROVAL_MODE = "https://www.facebook.com/messaging/set_approval_mode/?dpr=1"
-    CREATE_GROUP = "https://m.facebook.com/messages/send/?icm=1"
-    DELETE_THREAD = "https://www.facebook.com/ajax/mercury/delete_thread.php?dpr=1"
-    DELETE_MESSAGES = "https://www.facebook.com/ajax/mercury/delete_messages.php?dpr=1"
-    MUTE_THREAD = "https://www.facebook.com/ajax/mercury/change_mute_thread.php?dpr=1"
-    MUTE_REACTIONS = (
-        "https://www.facebook.com/ajax/mercury/change_reactions_mute_thread/?dpr=1"
-    )
-    MUTE_MENTIONS = (
-        "https://www.facebook.com/ajax/mercury/change_mentions_mute_thread/?dpr=1"
-    )
-    CREATE_POLL = "https://www.facebook.com/messaging/group_polling/create_poll/?dpr=1"
-    UPDATE_VOTE = "https://www.facebook.com/messaging/group_polling/update_vote/?dpr=1"
-    GET_POLL_OPTIONS = "https://www.facebook.com/ajax/mercury/get_poll_options"
-    SEARCH_MESSAGES = "https://www.facebook.com/ajax/mercury/search_snippets.php?dpr=1"
-    MARK_SPAM = "https://www.facebook.com/ajax/mercury/mark_spam.php?dpr=1"
-    UNSEND = "https://www.facebook.com/messaging/unsend_message/?dpr=1"
-
-    pull_channel = 0
-
-    def change_pull_channel(self, channel=None):
-        if channel is None:
-            self.pull_channel = (self.pull_channel + 1) % 5  # Pull channel will be 0-4
-        else:
-            self.pull_channel = channel
-        self.STICKY = "https://{}-edge-chat.facebook.com/pull".format(self.pull_channel)
-        self.PING = "https://{}-edge-chat.facebook.com/active_ping".format(
-            self.pull_channel
-        )
 
 
-facebookEncoding = "UTF-8"
+def strip_json_cruft(text: str) -> str:
-
+    """Removes `for(;;);` (and other cruft) that preceeds JSON responses."""
-
-def now():
-    return int(time() * 1000)
-
-
-def strip_to_json(text):
     try:
         return text[text.index("{") :]
-    except ValueError:
+    except ValueError as e:
-        raise FBchatException("No JSON object found: {!r}".format(text))
+        raise _exception.ParseError("No JSON object found", data=text) from e
 
 
-def get_decoded_r(r):
+def parse_json(text: str) -> Any:
-    return get_decoded(r._content)
+    try:
+        return json.loads(text)
+    except ValueError as e:
+        raise _exception.ParseError("Error while parsing JSON", data=text) from e
 
 
-def get_decoded(content):
+def generate_offline_threading_id():
-    return content.decode(facebookEncoding)
+    ret = datetime_to_millis(now())
-
+    value = int(random.random() * 4294967295)
-
-def parse_json(content):
-    return json.loads(content)
-
-
-def get_json(r):
-    return json.loads(strip_to_json(get_decoded_r(r)))
-
-
-def digitToChar(digit):
-    if digit < 10:
-        return str(digit)
-    return chr(ord("a") + digit - 10)
-
-
-def str_base(number, base):
-    if number < 0:
-        return "-" + str_base(-number, base)
-    (d, m) = divmod(number, base)
-    if d > 0:
-        return str_base(d, base) + digitToChar(m)
-    return digitToChar(m)
-
-
-def generateMessageID(client_id=None):
-    k = now()
-    l = int(random() * 4294967295)
-    return "<{}:{}-{}@mail.projektitan.com>".format(k, l, client_id)
-
-
-def getSignatureID():
-    return hex(int(random() * 2147483648))
-
-
-def generateOfflineThreadingID():
-    ret = now()
-    value = int(random() * 4294967295)
     string = ("0000000000000000000000" + format(value, "b"))[-22:]
     msgs = format(ret, "b") + string
     return str(int(msgs, 2))
 
 
-def check_json(j):
+def remove_version_from_module(module):
-    if j.get("error") is None:
+    return module.split("@", 1)[0]
-        return
-    if "errorDescription" in j:
-        # 'errorDescription' is in the users own language!
-        raise FBchatFacebookError(
-            "Error #{} when sending request: {}".format(
-                j["error"], j["errorDescription"]
-            ),
-            fb_error_code=j["error"],
-            fb_error_message=j["errorDescription"],
-        )
-    elif "debug_info" in j["error"] and "code" in j["error"]:
-        raise FBchatFacebookError(
-            "Error #{} when sending request: {}".format(
-                j["error"]["code"], repr(j["error"]["debug_info"])
-            ),
-            fb_error_code=j["error"]["code"],
-            fb_error_message=j["error"]["debug_info"],
-        )
-    else:
-        raise FBchatFacebookError(
-            "Error {} when sending request".format(j["error"]), fb_error_code=j["error"]
-        )
 
 
-def check_request(r, as_json=True):
+def get_jsmods_require(require) -> Mapping[str, Sequence[Any]]:
-    if not r.ok:
+    rtn = {}
-        raise FBchatFacebookError(
+    for item in require:
-            "Error when sending request: Got {} response".format(r.status_code),
+        if len(item) == 1:
-            request_status_code=r.status_code,
+            (module,) = item
-        )
+            rtn[remove_version_from_module(module)] = []
-
+            continue
-    content = get_decoded_r(r)
+        module, method, requirements, arguments = item
-
+        method = "{}.{}".format(remove_version_from_module(module), method)
-    if content is None or len(content) == 0:
+        rtn[method] = arguments
-        raise FBchatFacebookError("Error when sending request: Got empty response")
+    return rtn
-
-    if as_json:
-        content = strip_to_json(content)
-        try:
-            j = json.loads(content)
-        except ValueError:
-            raise FBchatFacebookError("Error while parsing JSON: {!r}".format(content))
-        check_json(j)
-        log.debug(j)
-        return j
-    else:
-        return content
 
 
-def get_jsmods_require(j, index):
+def get_jsmods_define(define) -> Mapping[str, Mapping[str, Any]]:
-    if j.get("jsmods") and j["jsmods"].get("require"):
+    rtn = {}
-        try:
+    for item in define:
-            return j["jsmods"]["require"][0][index][0]
+        module, requirements, data, _ = item
-        except (KeyError, IndexError) as e:
+        rtn[module] = data
-            log.warning(
+    return rtn
-                "Error when getting jsmods_require: {}. Facebook might have changed protocol".format(
-                    j
-                )
-            )
-    return None
 
 
-def get_emojisize_from_tags(tags):
+def mimetype_to_key(mimetype: str) -> str:
-    if tags is None:
-        return None
-    tmp = [tag for tag in tags if tag.startswith("hot_emoji_size:")]
-    if len(tmp) > 0:
-        try:
-            return LIKES[tmp[0].split(":")[1]]
-        except (KeyError, IndexError):
-            log.exception(
-                "Could not determine emoji size from {} - {}".format(tags, tmp)
-            )
-    return None
-
-
-def require_list(list_):
-    if isinstance(list_, list):
-        return set(list_)
-    else:
-        return set([list_])
-
-
-def mimetype_to_key(mimetype):
     if not mimetype:
         return "file_id"
     if mimetype == "image/gif":
@@ -327,51 +99,70 @@ def mimetype_to_key(mimetype):
     return "file_id"
 
 
-def get_files_from_urls(file_urls):
+def get_url_parameter(url: str, param: str) -> Optional[str]:
-    files = []
+    params = urllib.parse.parse_qs(urllib.parse.urlparse(url).query)
-    for file_url in file_urls:
+    if not params.get(param):
-        r = requests.get(file_url)
+        return None
-        # We could possibly use r.headers.get('Content-Disposition'), see
+    return params[param][0]
-        # https://stackoverflow.com/a/37060758
+
-        files.append(
+
-            (
+def seconds_to_datetime(timestamp_in_seconds: float) -> datetime.datetime:
-                basename(file_url),
+    """Convert an UTC timestamp to a timezone-aware datetime object."""
-                r.content,
+    # `.utcfromtimestamp` will return a "naive" datetime object, which is why we use the
-                r.headers.get("Content-Type") or guess_type(file_url)[0],
+    # following:
+    return datetime.datetime.fromtimestamp(
+        timestamp_in_seconds, tz=datetime.timezone.utc
     )
-        )
-    return files
 
 
-@contextmanager
+def millis_to_datetime(timestamp_in_milliseconds: int) -> datetime.datetime:
-def get_files_from_paths(filenames):
+    """Convert an UTC timestamp, in milliseconds, to a timezone-aware datetime."""
-    files = []
+    return seconds_to_datetime(timestamp_in_milliseconds / 1000)
-    for filename in filenames:
-        files.append(
-            (basename(filename), open(filename, "rb"), guess_type(filename)[0])
-        )
-    yield files
-    for fn, fp, ft in files:
-        fp.close()
 
 
-def enum_extend_if_invalid(enumeration, value):
+def datetime_to_seconds(dt: datetime.datetime) -> int:
-    try:
+    """Convert a datetime to an UTC timestamp.
-        return enumeration(value)
+
-    except ValueError:
+    Naive datetime objects are presumed to represent time in the system timezone.
-        log.warning(
+
-            "Failed parsing {.__name__}({!r}). Extending enum.".format(
+    The returned seconds will be rounded to the nearest whole number.
-                enumeration, value
+    """
-            )
+    # We could've implemented some fancy "convert naive timezones to UTC" logic, but
-        )
+    # it's not really worth the effort.
-        aenum.extend_enum(enumeration, "UNKNOWN_{}".format(value).upper(), value)
+    return round(dt.timestamp())
-        return enumeration(value)
 
 
-def get_url_parameters(url, *args):
+def datetime_to_millis(dt: datetime.datetime) -> int:
-    params = parse_qs(urlparse(url).query)
+    """Convert a datetime to an UTC timestamp, in milliseconds.
-    return [params[arg][0] for arg in args if params.get(arg)]
+
+    Naive datetime objects are presumed to represent time in the system timezone.
+
+    The returned milliseconds will be rounded to the nearest whole number.
+    """
+    return round(dt.timestamp() * 1000)
 
 
-def get_url_parameter(url, param):
+def seconds_to_timedelta(seconds: float) -> datetime.timedelta:
-    return get_url_parameters(url, param)[0]
+    """Convert seconds to a timedelta."""
+    return datetime.timedelta(seconds=seconds)
+
+
+def millis_to_timedelta(milliseconds: int) -> datetime.timedelta:
+    """Convert a duration (in milliseconds) to a timedelta object."""
+    return datetime.timedelta(milliseconds=milliseconds)
+
+
+def timedelta_to_seconds(td: datetime.timedelta) -> int:
+    """Convert a timedelta to seconds.
+
+    The returned seconds will be rounded to the nearest whole number.
+    """
+    return round(td.total_seconds())
+
+
+def now() -> datetime.datetime:
+    """The current time.
+
+    Similar to datetime.datetime.now(), but returns a non-naive datetime.
+    """
+    return datetime.datetime.now(tz=datetime.timezone.utc)

fbchat/client.py

@@ -1,8 +0,0 @@
-# -*- coding: UTF-8 -*-
-"""This file is here to maintain backwards compatability."""
-from __future__ import unicode_literals
-
-from .utils import *
-from .models import *
-from .graphql import *
-from ._client import Client

fbchat/graphql.py

@@ -1,31 +0,0 @@
-# -*- coding: UTF-8 -*-
-"""This file is here to maintain backwards compatability."""
-from __future__ import unicode_literals
-
-from .models import *
-from .utils import *
-from ._graphql import (
-    FLAGS,
-    WHITESPACE,
-    ConcatJSONDecoder,
-    graphql_color_to_enum,
-    get_customization_info,
-    graphql_to_sticker,
-    graphql_to_attachment,
-    graphql_to_extensible_attachment,
-    graphql_to_subattachment,
-    graphql_to_live_location,
-    graphql_to_poll,
-    graphql_to_poll_option,
-    graphql_to_plan,
-    graphql_to_quick_reply,
-    graphql_to_message,
-    graphql_to_message_reply,
-    graphql_to_user,
-    graphql_to_thread,
-    graphql_to_group,
-    graphql_to_page,
-    graphql_queries_to_json,
-    graphql_response_to_json,
-    GraphQL,
-)

fbchat/models.py

@@ -1,29 +0,0 @@
-# -*- coding: UTF-8 -*-
-"""This file is here to maintain backwards compatability, and to re-export our models
-into the global module (see `__init__.py`).
-
-A common pattern was to use `from fbchat.models import *`, hence we need this while
-transitioning to a better code structure.
-"""
-from __future__ import unicode_literals
-
-from ._core import Enum
-from ._exception import FBchatException, FBchatFacebookError, FBchatUserError
-from ._thread import ThreadType, ThreadLocation, ThreadColor, Thread
-from ._user import TypingStatus, User, ActiveStatus
-from ._group import Group, Room
-from ._page import Page
-from ._message import EmojiSize, MessageReaction, Mention, Message
-from ._attachment import Attachment, UnsentMessage, ShareAttachment
-from ._sticker import Sticker
-from ._location import LocationAttachment, LiveLocationAttachment
-from ._file import FileAttachment, AudioAttachment, ImageAttachment, VideoAttachment
-from ._quick_reply import (
-    QuickReply,
-    QuickReplyText,
-    QuickReplyLocation,
-    QuickReplyPhoneNumber,
-    QuickReplyEmail,
-)
-from ._poll import Poll, PollOption
-from ._plan import Plan

fbchat/utils.py

@@ -1,36 +0,0 @@
-# -*- coding: UTF-8 -*-
-"""This file is here to maintain backwards compatability."""
-from __future__ import unicode_literals
-
-from .models import *
-from ._util import (
-    log,
-    handler,
-    USER_AGENTS,
-    LIKES,
-    GENDERS,
-    ReqUrl,
-    facebookEncoding,
-    now,
-    strip_to_json,
-    get_decoded_r,
-    get_decoded,
-    parse_json,
-    get_json,
-    digitToChar,
-    str_base,
-    generateMessageID,
-    getSignatureID,
-    generateOfflineThreadingID,
-    check_json,
-    check_request,
-    get_jsmods_require,
-    get_emojisize_from_tags,
-    require_list,
-    mimetype_to_key,
-    get_files_from_urls,
-    get_files_from_paths,
-    enum_extend_if_invalid,
-    get_url_parameters,
-    get_url_parameter,
-)

pyproject.toml

@@ -1,14 +1,6 @@
 [tool.black]
 line-length = 88
-exclude = '''
+target-version = ['py36', 'py37', 'py38']
-/(
-    \.git
-  | \.pytest_cache
-  | build
-  | dist
-  | venv
-)/
-'''
 
 [build-system]
 requires = ["flit"]
@@ -22,10 +14,10 @@ maintainer = "Mads Marquart"
 maintainer-email = "madsmtm@gmail.com"
 home-page = "https://github.com/carpedm20/fbchat/"
 requires = [
-    "aenum",
+    "attrs>=19.1",
-    "attrs~=18.2.0",
+    "requests~=2.19",
-    "requests",
+    "beautifulsoup4~=4.0",
-    "beautifulsoup4",
+    "paho-mqtt~=1.5",
 ]
 description-file = "README.rst"
 classifiers = [
@@ -36,12 +28,12 @@ classifiers = [
     "Operating System :: OS Independent",
     "Natural Language :: English",
     "Programming Language :: Python",
-    "Programming Language :: Python :: 2.7",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.4",
+    "Programming Language :: Python :: 3 :: Only",
     "Programming Language :: Python :: 3.5",
     "Programming Language :: Python :: 3.6",
     "Programming Language :: Python :: 3.7",
+    "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: Implementation :: CPython",
     "Programming Language :: Python :: Implementation :: PyPy",
     "Topic :: Communications :: Chat",
@@ -50,7 +42,7 @@ classifiers = [
     "Topic :: Software Development :: Libraries",
     "Topic :: Software Development :: Libraries :: Python Modules",
 ]
-requires-python = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4.0"
+requires-python = ">=3.5, <4.0"
 keywords = "Facebook FB Messenger Library Chat Api Bot"
 license = "BSD 3-Clause"
 
@@ -60,6 +52,13 @@ Repository = "https://github.com/carpedm20/fbchat/"
 
 [tool.flit.metadata.requires-extra]
 test = [
-    "pytest~=4.0",
+    "pytest>=4.3,<6.0",
-    "six",
+]
+docs = [
+    "sphinx~=2.0",
+    "sphinxcontrib-spelling~=4.0",
+    "sphinx-autodoc-typehints~=1.10",
+]
+lint = [
+    "black",
 ]

pytest.ini

@@ -1,6 +1,10 @@
 [pytest]
 xfail_strict = true
 markers =
-    offline: Offline tests, aka. tests that can be executed without the need of a client
+    online: Online tests, that require a user account set up. Meant to be used \
-    expensive: Expensive tests, which should be executed sparingly
+    manually, to check whether Facebook has broken something.
-addopts = -m "not expensive"
+addopts =
+    --strict
+    -m "not online"
+testpaths = tests
+filterwarnings = error

tests/conftest.py

@@ -1,129 +1,9 @@
-# -*- coding: utf-8 -*-
-
-from __future__ import unicode_literals
-
 import pytest
-import json
+import fbchat
-
-from utils import *
-from contextlib import contextmanager
-from fbchat.models import ThreadType, Message, Mention
 
 
 @pytest.fixture(scope="session")
-def user(client2):
+def session():
-    return {"id": client2.uid, "type": ThreadType.USER}
+    return fbchat.Session(
-
+        user_id="31415926536", fb_dtsg=None, revision=None, session=None
-
-@pytest.fixture(scope="session")
-def group(pytestconfig):
-    return {
-        "id": load_variable("group_id", pytestconfig.cache),
-        "type": ThreadType.GROUP,
-    }
-
-
-@pytest.fixture(
-    scope="session",
-    params=["user", "group", pytest.param("none", marks=[pytest.mark.xfail()])],
     )
-def thread(request, user, group):
-    return {
-        "user": user,
-        "group": group,
-        "none": {"id": "0", "type": ThreadType.GROUP},
-    }[request.param]
-
-
-@pytest.fixture(scope="session")
-def client1(pytestconfig):
-    with load_client(1, pytestconfig.cache) as c:
-        yield c
-
-
-@pytest.fixture(scope="session")
-def client2(pytestconfig):
-    with load_client(2, pytestconfig.cache) as c:
-        yield c
-
-
-@pytest.fixture(scope="module")
-def client(client1, thread):
-    client1.setDefaultThread(thread["id"], thread["type"])
-    yield client1
-    client1.resetDefaultThread()
-
-
-@pytest.fixture(scope="session", params=["client1", "client2"])
-def client_all(request, client1, client2):
-    return client1 if request.param == "client1" else client2
-
-
-@pytest.fixture(scope="session")
-def catch_event(client2):
-    t = ClientThread(client2)
-    t.start()
-
-    @contextmanager
-    def inner(method_name):
-        caught = CaughtValue()
-        old_method = getattr(client2, method_name)
-
-        # Will be called by the other thread
-        def catch_value(*args, **kwargs):
-            old_method(*args, **kwargs)
-            # Make sure the `set` is only called once
-            if not caught.is_set():
-                caught.set(kwargs)
-
-        setattr(client2, method_name, catch_value)
-        yield caught
-        caught.wait()
-        if not caught.is_set():
-            raise ValueError("The value could not be caught")
-        setattr(client2, method_name, old_method)
-
-    yield inner
-
-    t.should_stop.set()
-
-    try:
-        # Make the client send a messages to itself, so the blocking pull request will return
-        # This is probably not safe, since the client is making two requests simultaneously
-        client2.sendMessage(random_hex(), client2.uid)
-    finally:
-        t.join()
-
-
-@pytest.fixture(scope="module")
-def compare(client, thread):
-    def inner(caught_event, **kwargs):
-        d = {
-            "author_id": client.uid,
-            "thread_id": client.uid
-            if thread["type"] == ThreadType.USER
-            else thread["id"],
-            "thread_type": thread["type"],
-        }
-        d.update(kwargs)
-        return subset(caught_event.res, **d)
-
-    return inner
-
-
-@pytest.fixture(params=["me", "other", "me other"])
-def message_with_mentions(request, client, client2, group):
-    text = "Hi there ["
-    mentions = []
-    if "me" in request.param:
-        mentions.append(Mention(thread_id=client.uid, offset=len(text), length=2))
-        text += "me, "
-    if "other" in request.param:
-        mentions.append(Mention(thread_id=client2.uid, offset=len(text), length=5))
-        text += "other, "
-    # Unused, because Facebook don't properly support sending mentions with groups as targets
-    if "group" in request.param:
-        mentions.append(Mention(thread_id=group["id"], offset=len(text), length=5))
-        text += "group, "
-    text += "nothing]"
-    return Message(text, mentions=mentions)

tests/events/test_client_payload.py

@@ -0,0 +1,175 @@
+import datetime
+import pytest
+from fbchat import (
+    ParseError,
+    User,
+    Group,
+    Message,
+    MessageData,
+    UnknownEvent,
+    ReactionEvent,
+    UserStatusEvent,
+    LiveLocationEvent,
+    UnsendEvent,
+    MessageReplyEvent,
+)
+from fbchat._events import parse_client_delta, parse_client_payloads
+
+
+def test_reaction_event_added(session):
+    data = {
+        "threadKey": {"otherUserFbId": 1234},
+        "messageId": "mid.$XYZ",
+        "action": 0,
+        "userId": 4321,
+        "reaction": "😍",
+        "senderId": 4321,
+        "offlineThreadingId": "6623596674408921967",
+    }
+    thread = User(session=session, id="1234")
+    assert ReactionEvent(
+        author=User(session=session, id="4321"),
+        thread=thread,
+        message=Message(thread=thread, id="mid.$XYZ"),
+        reaction="😍",
+    ) == parse_client_delta(session, {"deltaMessageReaction": data})
+
+
+def test_reaction_event_removed(session):
+    data = {
+        "threadKey": {"threadFbId": 1234},
+        "messageId": "mid.$XYZ",
+        "action": 1,
+        "userId": 4321,
+        "senderId": 4321,
+        "offlineThreadingId": "6623586106713014836",
+    }
+    thread = Group(session=session, id="1234")
+    assert ReactionEvent(
+        author=User(session=session, id="4321"),
+        thread=thread,
+        message=Message(thread=thread, id="mid.$XYZ"),
+        reaction=None,
+    ) == parse_client_delta(session, {"deltaMessageReaction": data})
+
+
+def test_user_status_blocked(session):
+    data = {
+        "threadKey": {"otherUserFbId": 1234},
+        "canViewerReply": False,
+        "reason": 2,
+        "actorFbid": 4321,
+    }
+    assert UserStatusEvent(
+        author=User(session=session, id="4321"),
+        thread=User(session=session, id="1234"),
+        blocked=True,
+    ) == parse_client_delta(session, {"deltaChangeViewerStatus": data})
+
+
+def test_user_status_unblocked(session):
+    data = {
+        "threadKey": {"otherUserFbId": 1234},
+        "canViewerReply": True,
+        "reason": 2,
+        "actorFbid": 1234,
+    }
+    assert UserStatusEvent(
+        author=User(session=session, id="1234"),
+        thread=User(session=session, id="1234"),
+        blocked=False,
+    ) == parse_client_delta(session, {"deltaChangeViewerStatus": data})
+
+
+@pytest.mark.skip(reason="need to gather test data")
+def test_live_location(session):
+    pass
+
+
+def test_message_reply(session):
+    message = {
+        "messageMetadata": {
+            "threadKey": {"otherUserFbId": 1234},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "112233445566",
+            "actorFbId": 1234,
+            "timestamp": 1500000000000,
+            "tags": ["source:messenger:web", "cg-enabled", "sent", "inbox"],
+            "threadReadStateEffect": 3,
+            "skipBumpThread": False,
+            "skipSnippetUpdate": False,
+            "unsendType": "can_unsend",
+            "folderId": {"systemFolderId": 0},
+        },
+        "body": "xyz",
+        "attachments": [],
+        "irisSeqId": 1111111,
+        "messageReply": {"replyToMessageId": {"id": "mid.$ABC"}, "status": 0,},
+        "requestContext": {"apiArgs": "..."},
+        "irisTags": ["DeltaNewMessage"],
+    }
+    reply = {
+        "messageMetadata": {
+            "threadKey": {"otherUserFbId": 1234},
+            "messageId": "mid.$ABC",
+            "offlineThreadingId": "665544332211",
+            "actorFbId": 4321,
+            "timestamp": 1600000000000,
+            "tags": ["inbox", "sent", "source:messenger:web"],
+        },
+        "body": "abc",
+        "attachments": [],
+        "requestContext": {"apiArgs": "..."},
+        "irisTags": [],
+    }
+    data = {
+        "message": message,
+        "repliedToMessage": reply,
+        "status": 0,
+    }
+    thread = User(session=session, id="1234")
+    assert MessageReplyEvent(
+        author=User(session=session, id="1234"),
+        thread=thread,
+        message=MessageData(
+            thread=thread,
+            id="mid.$XYZ",
+            author="1234",
+            created_at=datetime.datetime(
+                2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc
+            ),
+            text="xyz",
+            reply_to_id="mid.$ABC",
+        ),
+        replied_to=MessageData(
+            thread=thread,
+            id="mid.$ABC",
+            author="4321",
+            created_at=datetime.datetime(
+                2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc
+            ),
+            text="abc",
+        ),
+    ) == parse_client_delta(session, {"deltaMessageReply": data})
+
+
+def test_parse_client_delta_unknown(session):
+    assert UnknownEvent(
+        source="client payload", data={"abc": 10}
+    ) == parse_client_delta(session, {"abc": 10})
+
+
+def test_parse_client_payloads_empty(session):
+    # This is never something that happens, it's just so that we can test the parsing
+    # payload = '{"deltas":[]}'
+    payload = [123, 34, 100, 101, 108, 116, 97, 115, 34, 58, 91, 93, 125]
+    data = {"payload": payload, "class": "ClientPayload"}
+    assert [] == list(parse_client_payloads(session, data))
+
+
+def test_parse_client_payloads_invalid(session):
+    # payload = '{"invalid":"data"}'
+    payload = [123, 34, 105, 110, 118, 97, 108, 105, 100, 34, 58, 34, 97, 34, 125]
+    data = {"payload": payload, "class": "ClientPayload"}
+    with pytest.raises(ParseError, match="Error parsing ClientPayload"):
+        list(parse_client_payloads(session, data))

tests/events/test_common.py

@@ -0,0 +1,78 @@
+import pytest
+import datetime
+from fbchat import Group, User, ParseError, Event, ThreadEvent
+
+
+def test_event_get_thread_group1(session):
+    data = {
+        "threadKey": {"threadFbId": 1234},
+        "messageId": "mid.$gAAT4Sw1WSGh14A3MOFvrsiDvr3Yc",
+        "offlineThreadingId": "6623583531508397596",
+        "actorFbId": 4321,
+        "timestamp": 1500000000000,
+        "tags": [
+            "inbox",
+            "sent",
+            "tq",
+            "blindly_apply_message_folder",
+            "source:messenger:web",
+        ],
+    }
+    assert Group(session=session, id="1234") == Event._get_thread(session, data)
+
+
+def test_event_get_thread_group2(session):
+    data = {
+        "actorFbId": "4321",
+        "folderId": {"systemFolderId": "INBOX"},
+        "messageId": "mid.$XYZ",
+        "offlineThreadingId": "112233445566",
+        "skipBumpThread": False,
+        "tags": ["source:messenger:web"],
+        "threadKey": {"threadFbId": "1234"},
+        "threadReadStateEffect": "KEEP_AS_IS",
+        "timestamp": "1500000000000",
+    }
+    assert Group(session=session, id="1234") == Event._get_thread(session, data)
+
+
+def test_event_get_thread_user(session):
+    data = {
+        "actorFbId": "4321",
+        "folderId": {"systemFolderId": "INBOX"},
+        "messageId": "mid.$XYZ",
+        "offlineThreadingId": "112233445566",
+        "skipBumpThread": False,
+        "skipSnippetUpdate": False,
+        "tags": ["source:messenger:web"],
+        "threadKey": {"otherUserFbId": "1234"},
+        "threadReadStateEffect": "KEEP_AS_IS",
+        "timestamp": "1500000000000",
+    }
+    assert User(session=session, id="1234") == Event._get_thread(session, data)
+
+
+def test_event_get_thread_unknown(session):
+    data = {"threadKey": {"abc": "1234"}}
+    with pytest.raises(ParseError, match="Could not find thread data"):
+        Event._get_thread(session, data)
+
+
+def test_thread_event_parse_metadata(session):
+    data = {
+        "actorFbId": "4321",
+        "folderId": {"systemFolderId": "INBOX"},
+        "messageId": "mid.$XYZ",
+        "offlineThreadingId": "112233445566",
+        "skipBumpThread": False,
+        "skipSnippetUpdate": False,
+        "tags": ["source:messenger:web"],
+        "threadKey": {"otherUserFbId": "1234"},
+        "threadReadStateEffect": "KEEP_AS_IS",
+        "timestamp": "1500000000000",
+    }
+    assert (
+        User(session=session, id="4321"),
+        User(session=session, id="1234"),
+        datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == ThreadEvent._parse_metadata(session, {"messageMetadata": data})

tests/events/test_delta_class.py

@@ -0,0 +1,359 @@
+import datetime
+import pytest
+from fbchat import (
+    ParseError,
+    User,
+    Group,
+    Message,
+    MessageData,
+    ThreadLocation,
+    UnknownEvent,
+    PeopleAdded,
+    PersonRemoved,
+    TitleSet,
+    UnfetchedThreadEvent,
+    MessagesDelivered,
+    ThreadsRead,
+    MessageEvent,
+    ThreadFolder,
+)
+from fbchat._events import parse_delta
+
+
+def test_people_added(session):
+    data = {
+        "addedParticipants": [
+            {
+                "fanoutPolicy": "IRIS_MESSAGE_QUEUE",
+                "firstName": "Abc",
+                "fullName": "Abc Def",
+                "initialFolder": "FOLDER_INBOX",
+                "initialFolderId": {"systemFolderId": "INBOX"},
+                "isMessengerUser": False,
+                "userFbId": "1234",
+            }
+        ],
+        "irisSeqId": "11223344",
+        "irisTags": ["DeltaParticipantsAddedToGroupThread", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "3456",
+            "adminText": "You added Abc Def to the group.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "1122334455",
+            "skipBumpThread": False,
+            "tags": [],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456", "4567"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "class": "ParticipantsAddedToGroupThread",
+    }
+    assert PeopleAdded(
+        author=User(session=session, id="3456"),
+        thread=Group(session=session, id="4321"),
+        added=[User(session=session, id="1234")],
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_delta(session, data)
+
+
+def test_person_removed(session):
+    data = {
+        "irisSeqId": "11223344",
+        "irisTags": ["DeltaParticipantLeftGroupThread", "is_from_iris_fanout"],
+        "leftParticipantFbId": "1234",
+        "messageMetadata": {
+            "actorFbId": "3456",
+            "adminText": "You removed Abc Def from the group.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "1122334455",
+            "skipBumpThread": True,
+            "tags": [],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456", "4567"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "class": "ParticipantLeftGroupThread",
+    }
+    assert PersonRemoved(
+        author=User(session=session, id="3456"),
+        thread=Group(session=session, id="4321"),
+        removed=User(session=session, id="1234"),
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_delta(session, data)
+
+
+def test_title_set(session):
+    data = {
+        "irisSeqId": "11223344",
+        "irisTags": ["DeltaThreadName", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "3456",
+            "adminText": "You named the group abc.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "1122334455",
+            "skipBumpThread": False,
+            "tags": [],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "name": "abc",
+        "participants": ["1234", "2345", "3456", "4567"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "class": "ThreadName",
+    }
+    assert TitleSet(
+        author=User(session=session, id="3456"),
+        thread=Group(session=session, id="4321"),
+        title="abc",
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_delta(session, data)
+
+
+def test_title_removed(session):
+    data = {
+        "irisSeqId": "11223344",
+        "irisTags": ["DeltaThreadName", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "3456",
+            "adminText": "You removed the group name.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "1122334455",
+            "skipBumpThread": False,
+            "tags": [],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "name": "",
+        "participants": ["1234", "2345", "3456", "4567"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "class": "ThreadName",
+    }
+    assert TitleSet(
+        author=User(session=session, id="3456"),
+        thread=Group(session=session, id="4321"),
+        title=None,
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_delta(session, data)
+
+
+def test_forced_fetch(session):
+    data = {
+        "forceInsert": False,
+        "messageId": "mid.$XYZ",
+        "threadKey": {"threadFbId": "1234"},
+        "class": "ForcedFetch",
+    }
+    thread = Group(session=session, id="1234")
+    assert UnfetchedThreadEvent(
+        thread=thread, message=Message(thread=thread, id="mid.$XYZ")
+    ) == parse_delta(session, data)
+
+
+def test_forced_fetch_pending(session):
+    data = {
+        "forceInsert": False,
+        "irisSeqId": "1111",
+        "isLazy": False,
+        "threadKey": {"threadFbId": "1234"},
+        "class": "ForcedFetch",
+    }
+    assert UnfetchedThreadEvent(
+        thread=Group(session=session, id="1234"), message=None
+    ) == parse_delta(session, data)
+
+
+def test_delivery_receipt_group(session):
+    data = {
+        "actorFbId": "1234",
+        "deliveredWatermarkTimestampMs": "1500000000000",
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaDeliveryReceipt"],
+        "messageIds": ["mid.$XYZ", "mid.$ABC"],
+        "requestContext": {"apiArgs": {}},
+        "threadKey": {"threadFbId": "4321"},
+        "class": "DeliveryReceipt",
+    }
+    thread = Group(session=session, id="4321")
+    assert MessagesDelivered(
+        author=User(session=session, id="1234"),
+        thread=thread,
+        messages=[
+            Message(thread=thread, id="mid.$XYZ"),
+            Message(thread=thread, id="mid.$ABC"),
+        ],
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_delta(session, data)
+
+
+def test_delivery_receipt_user(session):
+    data = {
+        "deliveredWatermarkTimestampMs": "1500000000000",
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaDeliveryReceipt", "is_from_iris_fanout"],
+        "messageIds": ["mid.$XYZ", "mid.$ABC"],
+        "requestContext": {"apiArgs": {}},
+        "threadKey": {"otherUserFbId": "1234"},
+        "class": "DeliveryReceipt",
+    }
+    thread = User(session=session, id="1234")
+    assert MessagesDelivered(
+        author=thread,
+        thread=thread,
+        messages=[
+            Message(thread=thread, id="mid.$XYZ"),
+            Message(thread=thread, id="mid.$ABC"),
+        ],
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_delta(session, data)
+
+
+def test_read_receipt(session):
+    data = {
+        "actionTimestampMs": "1600000000000",
+        "actorFbId": "1234",
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaReadReceipt", "is_from_iris_fanout"],
+        "requestContext": {"apiArgs": {}},
+        "threadKey": {"threadFbId": "4321"},
+        "tqSeqId": "1111",
+        "watermarkTimestampMs": "1500000000000",
+        "class": "ReadReceipt",
+    }
+    assert ThreadsRead(
+        author=User(session=session, id="1234"),
+        threads=[Group(session=session, id="4321")],
+        at=datetime.datetime(2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_delta(session, data)
+
+
+def test_mark_read(session):
+    data = {
+        "actionTimestamp": "1600000000000",
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaMarkRead", "is_from_iris_fanout"],
+        "threadKeys": [{"threadFbId": "1234"}, {"otherUserFbId": "2345"}],
+        "tqSeqId": "1111",
+        "watermarkTimestamp": "1500000000000",
+        "class": "MarkRead",
+    }
+    assert ThreadsRead(
+        author=session.user,
+        threads=[Group(session=session, id="1234"), User(session=session, id="2345")],
+        at=datetime.datetime(2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_delta(session, data)
+
+
+def test_new_message_user(session):
+    data = {
+        "attachments": [],
+        "body": "test",
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaNewMessage"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "skipSnippetUpdate": False,
+            "tags": ["source:messenger:web"],
+            "threadKey": {"otherUserFbId": "1234"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1600000000000",
+        },
+        "requestContext": {"apiArgs": {}},
+        "class": "NewMessage",
+    }
+    assert MessageEvent(
+        author=User(session=session, id="1234"),
+        thread=User(session=session, id="1234"),
+        message=MessageData(
+            thread=User(session=session, id="1234"),
+            id="mid.$XYZ",
+            author="1234",
+            text="test",
+            created_at=datetime.datetime(
+                2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc
+            ),
+        ),
+        at=datetime.datetime(2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_delta(session, data)
+
+
+def test_new_message_group(session):
+    data = {
+        "attachments": [],
+        "body": "test",
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaNewMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "4321",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:messenger:web"],
+            "threadKey": {"threadFbId": "1234"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1600000000000",
+        },
+        "participants": ["4321", "5432", "6543"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "class": "NewMessage",
+    }
+    assert MessageEvent(
+        author=User(session=session, id="4321"),
+        thread=Group(session=session, id="1234"),
+        message=MessageData(
+            thread=Group(session=session, id="1234"),
+            id="mid.$XYZ",
+            author="4321",
+            text="test",
+            created_at=datetime.datetime(
+                2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc
+            ),
+        ),
+        at=datetime.datetime(2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_delta(session, data)
+
+
+def test_thread_folder(session):
+    data = {
+        "class": "ThreadFolder",
+        "folder": "FOLDER_PENDING",
+        "irisSeqId": "1111",
+        "irisTags": ["DeltaThreadFolder", "is_from_iris_fanout"],
+        "requestContext": {"apiArgs": {}},
+        "threadKey": {"otherUserFbId": "1234"},
+    }
+    assert ThreadFolder(
+        thread=User(session=session, id="1234"), folder=ThreadLocation.PENDING
+    ) == parse_delta(session, data)
+
+
+def test_noop(session):
+    assert parse_delta(session, {"class": "NoOp"}) is None
+
+
+def test_parse_delta_unknown(session):
+    data = {"class": "Abc"}
+    assert UnknownEvent(source="Delta class", data=data) == parse_delta(session, data)

tests/events/test_delta_type.py

@@ -0,0 +1,958 @@
+import datetime
+import pytest
+from fbchat import (
+    _util,
+    ParseError,
+    User,
+    Group,
+    Message,
+    MessageData,
+    Poll,
+    PollOption,
+    PlanData,
+    GuestStatus,
+    UnknownEvent,
+    ColorSet,
+    EmojiSet,
+    NicknameSet,
+    AdminsAdded,
+    AdminsRemoved,
+    ApprovalModeSet,
+    CallStarted,
+    CallEnded,
+    CallJoined,
+    PollCreated,
+    PollVoted,
+    PlanCreated,
+    PlanEnded,
+    PlanEdited,
+    PlanDeleted,
+    PlanResponded,
+)
+from fbchat._events import parse_admin_message
+
+
+def test_color_set(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You changed the chat theme to Orange.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web", "no_push"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "change_thread_theme",
+        "untypedData": {
+            "should_show_icon": "1",
+            "theme_color": "FFFF7E29",
+            "accessibility_label": "Orange",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert ColorSet(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        color="#ff7e29",
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_emoji_set(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You set the emoji to 🌟.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "skipSnippetUpdate": False,
+            "tags": ["source:generic_admin_text"],
+            "threadKey": {"otherUserFbId": "1234"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "requestContext": {"apiArgs": {}},
+        "type": "change_thread_icon",
+        "untypedData": {
+            "thread_icon_url": "https://www.facebook.com/images/emoji.php/v9/te0/1/16/1f31f.png",
+            "thread_icon": "🌟",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert EmojiSet(
+        author=User(session=session, id="1234"),
+        thread=User(session=session, id="1234"),
+        emoji="🌟",
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_nickname_set(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You set the nickname for Abc Def to abc.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web", "no_push"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "change_thread_nickname",
+        "untypedData": {"nickname": "abc", "participant_id": "2345"},
+        "class": "AdminTextMessage",
+    }
+    assert NicknameSet(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        subject=User(session=session, id="2345"),
+        nickname="abc",
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_nickname_clear(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You cleared your nickname.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "skipSnippetUpdate": False,
+            "tags": ["source:generic_admin_text"],
+            "threadKey": {"otherUserFbId": "1234"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "requestContext": {"apiArgs": {}},
+        "type": "change_thread_nickname",
+        "untypedData": {"nickname": "", "participant_id": "1234"},
+        "class": "AdminTextMessage",
+    }
+    assert NicknameSet(
+        author=User(session=session, id="1234"),
+        thread=User(session=session, id="1234"),
+        subject=User(session=session, id="1234"),
+        nickname=None,
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_admins_added(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You added Abc Def as a group admin.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": True,
+            "tags": ["source:titan:web"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "change_thread_admins",
+        "untypedData": {
+            "THREAD_CATEGORY": "GROUP",
+            "TARGET_ID": "2345",
+            "ADMIN_TYPE": "0",
+            "ADMIN_EVENT": "add_admin",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert AdminsAdded(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        added=[User(session=session, id="2345")],
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_admins_removed(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You removed yourself as a group admin.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": True,
+            "tags": ["source:titan:web"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "change_thread_admins",
+        "untypedData": {
+            "THREAD_CATEGORY": "GROUP",
+            "TARGET_ID": "1234",
+            "ADMIN_TYPE": "0",
+            "ADMIN_EVENT": "remove_admin",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert AdminsRemoved(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        removed=[User(session=session, id="1234")],
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_approvalmode_set(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You turned on member approval and will review requests to join the group.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": True,
+            "tags": ["source:titan:web", "no_push"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "change_thread_approval_mode",
+        "untypedData": {"APPROVAL_MODE": "1", "THREAD_CATEGORY": "GROUP"},
+        "class": "AdminTextMessage",
+    }
+    assert ApprovalModeSet(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        require_admin_approval=True,
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_approvalmode_unset(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You turned off member approval. Anyone with the link can join the group.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": True,
+            "tags": ["source:titan:web", "no_push"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "change_thread_approval_mode",
+        "untypedData": {"APPROVAL_MODE": "0", "THREAD_CATEGORY": "GROUP"},
+        "class": "AdminTextMessage",
+    }
+    assert ApprovalModeSet(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        require_admin_approval=False,
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_call_started(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You started a call.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web", "no_push"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "messenger_call_log",
+        "untypedData": {
+            "call_capture_attachments": "",
+            "caller_id": "1234",
+            "conference_name": "MESSENGER:134845267536444",
+            "rating": "",
+            "messenger_call_instance_id": "0",
+            "video": "",
+            "event": "group_call_started",
+            "server_info": "XYZ123ABC",
+            "call_duration": "0",
+            "callee_id": "0",
+        },
+        "class": "AdminTextMessage",
+    }
+    data2 = {
+        "callState": "AUDIO_GROUP_CALL",
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": [],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+        },
+        "serverInfoData": "XYZ123ABC",
+        "class": "RtcCallData",
+    }
+    assert CallStarted(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_group_call_ended(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "The call ended.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web", "no_push"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "messenger_call_log",
+        "untypedData": {
+            "call_capture_attachments": "",
+            "caller_id": "1234",
+            "conference_name": "MESSENGER:1234567890",
+            "rating": "0",
+            "messenger_call_instance_id": "1234567890",
+            "video": "",
+            "event": "group_call_ended",
+            "server_info": "XYZ123ABC",
+            "call_duration": "31",
+            "callee_id": "0",
+        },
+        "class": "AdminTextMessage",
+    }
+    data2 = {
+        "callState": "NO_ONGOING_CALL",
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": [],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+        },
+        "class": "RtcCallData",
+    }
+    assert CallEnded(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        duration=datetime.timedelta(seconds=31),
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_user_call_ended(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "Abc called you.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "skipSnippetUpdate": False,
+            "tags": ["source:generic_admin_text", "no_push"],
+            "threadKey": {"otherUserFbId": "1234"},
+            "threadReadStateEffect": "KEEP_AS_IS",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "requestContext": {"apiArgs": {}},
+        "type": "messenger_call_log",
+        "untypedData": {
+            "call_capture_attachments": "",
+            "caller_id": "1234",
+            "conference_name": "MESSENGER:1234567890",
+            "rating": "0",
+            "messenger_call_instance_id": "1234567890",
+            "video": "",
+            "event": "one_on_one_call_ended",
+            "server_info": "",
+            "call_duration": "3",
+            "callee_id": "100002950119740",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert CallEnded(
+        author=User(session=session, id="1234"),
+        thread=User(session=session, id="1234"),
+        duration=datetime.timedelta(seconds=3),
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_call_joined(session):
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "Abc joined the call.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "participant_joined_group_call",
+        "untypedData": {
+            "server_info_data": "XYZ123ABC",
+            "group_call_type": "0",
+            "joining_user": "2345",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert CallJoined(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_poll_created(session):
+    poll_data = {
+        "id": "112233",
+        "text": "A poll",
+        "total_count": 2,
+        "viewer_has_voted": "true",
+        "options": [
+            {
+                "id": "1001",
+                "text": "Option A",
+                "total_count": 1,
+                "viewer_has_voted": "true",
+                "voters": ["1234"],
+            },
+            {
+                "id": "1002",
+                "text": "Option B",
+                "total_count": 0,
+                "viewer_has_voted": "false",
+                "voters": [],
+            },
+        ],
+    }
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You created a poll: A poll.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "group_poll",
+        "untypedData": {
+            "added_option_ids": "[]",
+            "removed_option_ids": "[]",
+            "question_json": _util.json_minimal(poll_data),
+            "event_type": "question_creation",
+            "question_id": "112233",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert PollCreated(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        poll=Poll(
+            session=session,
+            id="112233",
+            question="A poll",
+            options=[
+                PollOption(
+                    id="1001",
+                    text="Option A",
+                    vote=True,
+                    voters=["1234"],
+                    votes_count=1,
+                ),
+                PollOption(
+                    id="1002", text="Option B", vote=False, voters=[], votes_count=0
+                ),
+            ],
+            options_count=2,
+        ),
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_poll_answered(session):
+    poll_data = {
+        "id": "112233",
+        "text": "A poll",
+        "total_count": 3,
+        "viewer_has_voted": "true",
+        "options": [
+            {
+                "id": "1002",
+                "text": "Option B",
+                "total_count": 2,
+                "viewer_has_voted": "true",
+                "voters": ["1234", "2345"],
+            },
+            {
+                "id": "1003",
+                "text": "Option C",
+                "total_count": 1,
+                "viewer_has_voted": "true",
+                "voters": ["1234"],
+            },
+            {
+                "id": "1001",
+                "text": "Option A",
+                "total_count": 0,
+                "viewer_has_voted": "false",
+                "voters": [],
+            },
+        ],
+    }
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": 'You changed your vote to "Option B" and 1 other option in the poll: A poll.',
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "group_poll",
+        "untypedData": {
+            "added_option_ids": "[1002,1003]",
+            "removed_option_ids": "[1001]",
+            "question_json": _util.json_minimal(poll_data),
+            "event_type": "update_vote",
+            "question_id": "112233",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert PollVoted(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        poll=Poll(
+            session=session,
+            id="112233",
+            question="A poll",
+            options=[
+                PollOption(
+                    id="1002",
+                    text="Option B",
+                    vote=True,
+                    voters=["1234", "2345"],
+                    votes_count=2,
+                ),
+                PollOption(
+                    id="1003",
+                    text="Option C",
+                    vote=True,
+                    voters=["1234"],
+                    votes_count=1,
+                ),
+                PollOption(
+                    id="1001", text="Option A", vote=False, voters=[], votes_count=0
+                ),
+            ],
+            options_count=3,
+        ),
+        added_ids=["1002", "1003"],
+        removed_ids=["1001"],
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_plan_created(session):
+    guest_list = [
+        {"guest_list_state": "INVITED", "node": {"id": "3456"}},
+        {"guest_list_state": "INVITED", "node": {"id": "2345"}},
+        {"guest_list_state": "GOING", "node": {"id": "1234"}},
+    ]
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You created a plan.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web", "no_push"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "lightweight_event_create",
+        "untypedData": {
+            "event_timezone": "",
+            "event_creator_id": "1234",
+            "event_id": "112233",
+            "event_type": "EVENT",
+            "event_track_rsvp": "1",
+            "event_title": "A plan",
+            "event_time": "1600000000",
+            "event_seconds_to_notify_before": "3600",
+            "guest_state_list": _util.json_minimal(guest_list),
+        },
+        "class": "AdminTextMessage",
+    }
+    assert PlanCreated(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        plan=PlanData(
+            session=session,
+            id="112233",
+            time=datetime.datetime(
+                2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc
+            ),
+            title="A plan",
+            author_id="1234",
+            guests={
+                "1234": GuestStatus.GOING,
+                "2345": GuestStatus.INVITED,
+                "3456": GuestStatus.INVITED,
+            },
+        ),
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+@pytest.mark.skip(reason="Need to gather test data")
+def test_plan_ended(session):
+    data = {}
+    assert PlanEnded(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        plan=PlanData(
+            session=session,
+            id="112233",
+            time=datetime.datetime(
+                2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc
+            ),
+            title="A plan",
+            author_id="1234",
+            guests={
+                "1234": GuestStatus.GOING,
+                "2345": GuestStatus.INVITED,
+                "3456": GuestStatus.INVITED,
+            },
+        ),
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_plan_edited(session):
+    guest_list = [
+        {"guest_list_state": "INVITED", "node": {"id": "3456"}},
+        {"guest_list_state": "INVITED", "node": {"id": "2345"}},
+        {"guest_list_state": "GOING", "node": {"id": "1234"}},
+    ]
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You named the plan A plan.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web", "no_push"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "lightweight_event_update",
+        "untypedData": {
+            "event_creator_id": "1234",
+            "latitude": "0",
+            "event_title": "A plan",
+            "event_seconds_to_notify_before": "3600",
+            "guest_state_list": _util.json_minimal(guest_list),
+            "event_end_time": "0",
+            "event_timezone": "",
+            "event_id": "112233",
+            "event_type": "EVENT",
+            "event_location_id": "2233445566",
+            "event_location_name": "",
+            "event_time": "1600000000",
+            "event_note": "",
+            "longitude": "0",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert PlanEdited(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        plan=PlanData(
+            session=session,
+            id="112233",
+            time=datetime.datetime(
+                2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc
+            ),
+            title="A plan",
+            location_id="2233445566",
+            author_id="1234",
+            guests={
+                "1234": GuestStatus.GOING,
+                "2345": GuestStatus.INVITED,
+                "3456": GuestStatus.INVITED,
+            },
+        ),
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_plan_deleted(session):
+    guest_list = [
+        {"guest_list_state": "GOING", "node": {"id": "1234"}},
+        {"guest_list_state": "INVITED", "node": {"id": "3456"}},
+        {"guest_list_state": "INVITED", "node": {"id": "2345"}},
+    ]
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You deleted the plan A plan for Mon, 20 Jan at 15:30.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web", "no_push"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "lightweight_event_delete",
+        "untypedData": {
+            "event_end_time": "0",
+            "event_timezone": "",
+            "event_id": "112233",
+            "event_type": "EVENT",
+            "event_location_id": "2233445566",
+            "latitude": "0",
+            "event_title": "A plan",
+            "event_time": "1600000000",
+            "event_seconds_to_notify_before": "3600",
+            "guest_state_list": _util.json_minimal(guest_list),
+            "event_note": "",
+            "longitude": "0",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert PlanDeleted(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        plan=PlanData(
+            session=session,
+            id="112233",
+            time=datetime.datetime(
+                2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc
+            ),
+            title="A plan",
+            location_id="2233445566",
+            author_id=None,
+            guests={
+                "1234": GuestStatus.GOING,
+                "2345": GuestStatus.INVITED,
+                "3456": GuestStatus.INVITED,
+            },
+        ),
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_plan_participation(session):
+    guest_list = [
+        {"guest_list_state": "DECLINED", "node": {"id": "1234"}},
+        {"guest_list_state": "GOING", "node": {"id": "2345"}},
+        {"guest_list_state": "INVITED", "node": {"id": "3456"}},
+    ]
+    data = {
+        "irisSeqId": "1111111",
+        "irisTags": ["DeltaAdminTextMessage", "is_from_iris_fanout"],
+        "messageMetadata": {
+            "actorFbId": "1234",
+            "adminText": "You responded Can't Go to def.",
+            "folderId": {"systemFolderId": "INBOX"},
+            "messageId": "mid.$XYZ",
+            "offlineThreadingId": "11223344556677889900",
+            "skipBumpThread": False,
+            "tags": ["source:titan:web", "no_push"],
+            "threadKey": {"threadFbId": "4321"},
+            "threadReadStateEffect": "MARK_UNREAD",
+            "timestamp": "1500000000000",
+            "unsendType": "deny_log_message",
+        },
+        "participants": ["1234", "2345", "3456"],
+        "requestContext": {"apiArgs": {}},
+        "tqSeqId": "1111",
+        "type": "lightweight_event_rsvp",
+        "untypedData": {
+            "event_creator_id": "2345",
+            "guest_status": "DECLINED",
+            "latitude": "0",
+            "event_track_rsvp": "1",
+            "event_title": "A plan",
+            "event_seconds_to_notify_before": "3600",
+            "guest_state_list": _util.json_minimal(guest_list),
+            "event_end_time": "0",
+            "event_timezone": "",
+            "event_id": "112233",
+            "event_type": "EVENT",
+            "guest_id": "1234",
+            "event_location_id": "2233445566",
+            "event_time": "1600000000",
+            "event_note": "",
+            "longitude": "0",
+        },
+        "class": "AdminTextMessage",
+    }
+    assert PlanResponded(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        plan=PlanData(
+            session=session,
+            id="112233",
+            time=datetime.datetime(
+                2020, 9, 13, 12, 26, 40, tzinfo=datetime.timezone.utc
+            ),
+            title="A plan",
+            location_id="2233445566",
+            author_id="2345",
+            guests={
+                "1234": GuestStatus.DECLINED,
+                "2345": GuestStatus.GOING,
+                "3456": GuestStatus.INVITED,
+            },
+        ),
+        take_part=False,
+        at=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+    ) == parse_admin_message(session, data)
+
+
+def test_parse_admin_message_unknown(session):
+    data = {"class": "AdminTextMessage", "type": "abc"}
+    assert UnknownEvent(source="Delta type", data=data) == parse_admin_message(
+        session, data
+    )

tests/events/test_main.py

@@ -0,0 +1,137 @@
+import datetime
+from fbchat import (
+    _util,
+    User,
+    Group,
+    Message,
+    ParseError,
+    UnknownEvent,
+    Typing,
+    FriendRequest,
+    Presence,
+    ReactionEvent,
+    UnfetchedThreadEvent,
+    ActiveStatus,
+)
+from fbchat._events import parse_events
+
+
+def test_t_ms_full(session):
+    """A full example of parsing of data in /t_ms."""
+    payload = {
+        "deltas": [
+            {
+                "deltaMessageReaction": {
+                    "threadKey": {"threadFbId": 4321},
+                    "messageId": "mid.$XYZ",
+                    "action": 0,
+                    "userId": 1234,
+                    "reaction": "😢",
+                    "senderId": 1234,
+                    "offlineThreadingId": "1122334455",
+                }
+            }
+        ]
+    }
+    data = {
+        "deltas": [
+            {
+                "payload": [ord(x) for x in _util.json_minimal(payload)],
+                "class": "ClientPayload",
+            },
+            {"class": "NoOp",},
+            {
+                "forceInsert": False,
+                "messageId": "mid.$ABC",
+                "threadKey": {"threadFbId": "4321"},
+                "class": "ForcedFetch",
+            },
+        ],
+        "firstDeltaSeqId": 111111,
+        "lastIssuedSeqId": 111113,
+        "queueEntityId": 1234,
+    }
+    thread = Group(session=session, id="4321")
+    assert [
+        ReactionEvent(
+            author=User(session=session, id="1234"),
+            thread=thread,
+            message=Message(thread=thread, id="mid.$XYZ"),
+            reaction="😢",
+        ),
+        UnfetchedThreadEvent(
+            thread=thread, message=Message(thread=thread, id="mid.$ABC"),
+        ),
+    ] == list(parse_events(session, "/t_ms", data))
+
+
+def test_thread_typing(session):
+    data = {"sender_fbid": 1234, "state": 0, "type": "typ", "thread": "4321"}
+    (event,) = parse_events(session, "/thread_typing", data)
+    assert event == Typing(
+        author=User(session=session, id="1234"),
+        thread=Group(session=session, id="4321"),
+        status=False,
+    )
+
+
+def test_orca_typing_notifications(session):
+    data = {"type": "typ", "sender_fbid": 1234, "state": 1}
+    (event,) = parse_events(session, "/orca_typing_notifications", data)
+    assert event == Typing(
+        author=User(session=session, id="1234"),
+        thread=User(session=session, id="1234"),
+        status=True,
+    )
+
+
+def test_friend_request(session):
+    data = {"type": "jewel_requests_add", "from": "1234"}
+    (event,) = parse_events(session, "/legacy_web", data)
+    assert event == FriendRequest(author=User(session=session, id="1234"))
+
+
+def test_orca_presence_inc(session):
+    data = {
+        "list_type": "inc",
+        "list": [
+            {"u": 1234, "p": 0, "l": 1500000000, "vc": 74},
+            {"u": 2345, "p": 2, "c": 9969664, "vc": 10},
+        ],
+    }
+    (event,) = parse_events(session, "/orca_presence", data)
+    la = datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc)
+    assert event == Presence(
+        statuses={
+            "1234": ActiveStatus(active=False, last_active=la),
+            "2345": ActiveStatus(active=True),
+        },
+        full=False,
+    )
+
+
+def test_orca_presence_full(session):
+    data = {
+        "list_type": "full",
+        "list": [
+            {"u": 1234, "p": 2, "c": 5767242},
+            {"u": 2345, "p": 2, "l": 1500000000},
+            {"u": 3456, "p": 2, "c": 9961482},
+            {"u": 4567, "p": 0, "l": 1500000000},
+            {"u": 5678, "p": 0},
+            {"u": 6789, "p": 2, "c": 14168154},
+        ],
+    }
+    (event,) = parse_events(session, "/orca_presence", data)
+    la = datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc)
+    assert event == Presence(
+        statuses={
+            "1234": ActiveStatus(active=True),
+            "2345": ActiveStatus(active=True, last_active=la),
+            "3456": ActiveStatus(active=True),
+            "4567": ActiveStatus(active=False, last_active=la),
+            "5678": ActiveStatus(active=False),
+            "6789": ActiveStatus(active=True),
+        },
+        full=True,
+    )

tests/models/test_attachment.py

@@ -0,0 +1,459 @@
+import pytest
+import datetime
+import fbchat
+from fbchat import Image, UnsentMessage, ShareAttachment
+from fbchat._models._message import graphql_to_extensible_attachment
+
+
+def test_parse_unsent_message():
+    data = {
+        "legacy_attachment_id": "ee.mid.$xyz",
+        "story_attachment": {
+            "description": {"text": "You removed a message"},
+            "media": None,
+            "source": None,
+            "style_list": ["globally_deleted_message_placeholder", "fallback"],
+            "title_with_entities": {"text": ""},
+            "properties": [],
+            "url": None,
+            "deduplication_key": "deadbeef123",
+            "action_links": [],
+            "messaging_attribution": None,
+            "messenger_call_to_actions": [],
+            "xma_layout_info": None,
+            "target": None,
+            "subattachments": [],
+        },
+        "genie_attachment": {"genie_message": None},
+    }
+    assert UnsentMessage(id="ee.mid.$xyz") == graphql_to_extensible_attachment(data)
+
+
+def test_share_from_graphql_minimal():
+    data = {
+        "target": {},
+        "url": "a.com",
+        "title_with_entities": {"text": "a.com"},
+        "subattachments": [],
+    }
+    assert ShareAttachment(
+        url="a.com", original_url="a.com", title="a.com"
+    ) == ShareAttachment._from_graphql(data)
+
+
+def test_share_from_graphql_link():
+    data = {
+        "description": {"text": ""},
+        "media": {
+            "animated_image": None,
+            "image": None,
+            "playable_duration_in_ms": 0,
+            "is_playable": False,
+            "playable_url": None,
+        },
+        "source": {"text": "a.com"},
+        "style_list": ["share", "fallback"],
+        "title_with_entities": {"text": "a.com"},
+        "properties": [],
+        "url": "http://l.facebook.com/l.php?u=http%3A%2F%2Fa.com%2F&h=def&s=1",
+        "deduplication_key": "ee.mid.$xyz",
+        "action_links": [{"title": "About this website", "url": None}],
+        "messaging_attribution": None,
+        "messenger_call_to_actions": [],
+        "xma_layout_info": None,
+        "target": {"__typename": "ExternalUrl"},
+        "subattachments": [],
+    }
+    assert ShareAttachment(
+        author=None,
+        url="http://l.facebook.com/l.php?u=http%3A%2F%2Fa.com%2F&h=def&s=1",
+        original_url="http://a.com/",
+        title="a.com",
+        description="",
+        source="a.com",
+        image=None,
+        original_image_url=None,
+        attachments=[],
+        id="ee.mid.$xyz",
+    ) == ShareAttachment._from_graphql(data)
+
+
+def test_share_from_graphql_link_with_image():
+    data = {
+        "description": {
+            "text": (
+                "Create an account or log in to Facebook."
+                " Connect with friends, family and other people you know."
+                " Share photos and videos, send messages and get updates."
+            )
+        },
+        "media": {
+            "animated_image": None,
+            "image": {
+                "uri": "https://www.facebook.com/rsrc.php/v3/x.png",
+                "height": 325,
+                "width": 325,
+            },
+            "playable_duration_in_ms": 0,
+            "is_playable": False,
+            "playable_url": None,
+        },
+        "source": None,
+        "style_list": ["share", "fallback"],
+        "title_with_entities": {"text": "Facebook – log in or sign up"},
+        "properties": [],
+        "url": "http://facebook.com/",
+        "deduplication_key": "deadbeef123",
+        "action_links": [],
+        "messaging_attribution": None,
+        "messenger_call_to_actions": [],
+        "xma_layout_info": None,
+        "target": {"__typename": "ExternalUrl"},
+        "subattachments": [],
+    }
+    assert ShareAttachment(
+        author=None,
+        url="http://facebook.com/",
+        original_url="http://facebook.com/",
+        title="Facebook – log in or sign up",
+        description=(
+            "Create an account or log in to Facebook."
+            " Connect with friends, family and other people you know."
+            " Share photos and videos, send messages and get updates."
+        ),
+        source=None,
+        image=Image(
+            url="https://www.facebook.com/rsrc.php/v3/x.png", width=325, height=325
+        ),
+        original_image_url="https://www.facebook.com/rsrc.php/v3/x.png",
+        attachments=[],
+        id="deadbeef123",
+    ) == ShareAttachment._from_graphql(data)
+
+
+def test_share_from_graphql_video():
+    data = {
+        "description": {
+            "text": (
+                "Rick Astley's official music video for “Never Gonna Give You Up”"
+                " Listen to Rick Astley: https://RickAstley.lnk.to/_listenYD"
+                " Subscribe to the official Rick As..."
+            )
+        },
+        "media": {
+            "animated_image": None,
+            "image": {
+                "uri": (
+                    "https://external-arn2-1.xx.fbcdn.net/safe_image.php?d=xyz123"
+                    "&w=960&h=540&url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FdQw4w9WgXcQ"
+                    "%2Fmaxresdefault.jpg&sx=0&sy=0&sw=1280&sh=720&_nc_hash=abc123"
+                ),
+                "height": 540,
+                "width": 960,
+            },
+            "playable_duration_in_ms": 0,
+            "is_playable": True,
+            "playable_url": "https://www.youtube.com/embed/dQw4w9WgXcQ?autoplay=1",
+        },
+        "source": {"text": "youtube.com"},
+        "style_list": ["share", "fallback"],
+        "title_with_entities": {
+            "text": "Rick Astley - Never Gonna Give You Up (Video)"
+        },
+        "properties": [
+            {"key": "width", "value": {"text": "1280"}},
+            {"key": "height", "value": {"text": "720"}},
+        ],
+        "url": "https://l.facebook.com/l.php?u=https%3A%2F%2Fyoutu.be%2FdQw4w9WgXcQ",
+        "deduplication_key": "ee.mid.$gAAT4Sw1WSGhzQ9uRWVtEpZHZ8ZPV",
+        "action_links": [{"title": "About this website", "url": None}],
+        "messaging_attribution": None,
+        "messenger_call_to_actions": [],
+        "xma_layout_info": None,
+        "target": {"__typename": "ExternalUrl"},
+        "subattachments": [],
+    }
+    assert ShareAttachment(
+        author=None,
+        url="https://l.facebook.com/l.php?u=https%3A%2F%2Fyoutu.be%2FdQw4w9WgXcQ",
+        original_url="https://youtu.be/dQw4w9WgXcQ",
+        title="Rick Astley - Never Gonna Give You Up (Video)",
+        description=(
+            "Rick Astley's official music video for “Never Gonna Give You Up”"
+            " Listen to Rick Astley: https://RickAstley.lnk.to/_listenYD"
+            " Subscribe to the official Rick As..."
+        ),
+        source="youtube.com",
+        image=Image(
+            url="https://external-arn2-1.xx.fbcdn.net/safe_image.php?d=xyz123"
+            "&w=960&h=540&url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FdQw4w9WgXcQ"
+            "%2Fmaxresdefault.jpg&sx=0&sy=0&sw=1280&sh=720&_nc_hash=abc123",
+            width=960,
+            height=540,
+        ),
+        original_image_url="https://i.ytimg.com/vi/dQw4w9WgXcQ/maxresdefault.jpg",
+        attachments=[],
+        id="ee.mid.$gAAT4Sw1WSGhzQ9uRWVtEpZHZ8ZPV",
+    ) == ShareAttachment._from_graphql(data)
+
+
+def test_share_with_image_subattachment():
+    data = {
+        "description": {"text": "Abc"},
+        "media": {
+            "animated_image": None,
+            "image": {
+                "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t1.0-9/1.jpg",
+                "height": 960,
+                "width": 720,
+            },
+            "playable_duration_in_ms": 0,
+            "is_playable": False,
+            "playable_url": None,
+        },
+        "source": {"text": "Def"},
+        "style_list": ["attached_story", "fallback"],
+        "title_with_entities": {"text": ""},
+        "properties": [],
+        "url": "https://www.facebook.com/groups/11223344/permalink/1234/",
+        "deduplication_key": "deadbeef123",
+        "action_links": [
+            {"title": None, "url": None},
+            {"title": None, "url": "https://www.facebook.com/groups/11223344/"},
+            {
+                "title": "Report Post to Admin",
+                "url": "https://www.facebook.com/groups/11223344/members/",
+            },
+        ],
+        "messaging_attribution": None,
+        "messenger_call_to_actions": [],
+        "xma_layout_info": None,
+        "target": {
+            "__typename": "Story",
+            "title": None,
+            "description": {"text": "Abc"},
+            "actors": [
+                {
+                    "__typename": "User",
+                    "name": "Def",
+                    "id": "1111",
+                    "short_name": "Def",
+                    "url": "https://www.facebook.com/some-user",
+                    "profile_picture": {
+                        "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t1.0-1/c123.123.123.123a/s50x50/img.jpg",
+                        "height": 50,
+                        "width": 50,
+                    },
+                }
+            ],
+            "to": {
+                "__typename": "Group",
+                "name": "Some group",
+                "url": "https://www.facebook.com/groups/11223344/",
+            },
+            "attachments": [
+                {
+                    "url": "https://www.facebook.com/photo.php?fbid=4321&set=gm.1234&type=3",
+                    "media": {
+                        "is_playable": False,
+                        "image": {
+                            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t1.0-9/1.jpg",
+                            "height": 960,
+                            "width": 720,
+                        },
+                    },
+                }
+            ],
+            "attached_story": None,
+        },
+        "subattachments": [
+            {
+                "description": {"text": "Abc"},
+                "media": {
+                    "animated_image": None,
+                    "image": {
+                        "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t1.0-9/1.jpg",
+                        "height": 960,
+                        "width": 720,
+                    },
+                    "playable_duration_in_ms": 0,
+                    "is_playable": False,
+                    "playable_url": None,
+                },
+                "source": None,
+                "style_list": ["photo", "games_app", "fallback"],
+                "title_with_entities": {"text": ""},
+                "properties": [
+                    {"key": "photoset_reference_token", "value": {"text": "gm.1234"}},
+                    {"key": "layout_x", "value": {"text": "0"}},
+                    {"key": "layout_y", "value": {"text": "0"}},
+                    {"key": "layout_w", "value": {"text": "0"}},
+                    {"key": "layout_h", "value": {"text": "0"}},
+                ],
+                "url": "https://www.facebook.com/photo.php?fbid=4321&set=gm.1234&type=3",
+                "deduplication_key": "deadbeef456",
+                "action_links": [],
+                "messaging_attribution": None,
+                "messenger_call_to_actions": [],
+                "xma_layout_info": None,
+                "target": {"__typename": "Photo"},
+            }
+        ],
+    }
+    assert ShareAttachment(
+        author="1111",
+        url="https://www.facebook.com/groups/11223344/permalink/1234/",
+        original_url="https://www.facebook.com/groups/11223344/permalink/1234/",
+        title="",
+        description="Abc",
+        source="Def",
+        image=Image(
+            url="https://scontent-arn2-1.xx.fbcdn.net/v/t1.0-9/1.jpg",
+            width=720,
+            height=960,
+        ),
+        original_image_url="https://scontent-arn2-1.xx.fbcdn.net/v/t1.0-9/1.jpg",
+        attachments=[None],
+        id="deadbeef123",
+    ) == ShareAttachment._from_graphql(data)
+
+
+def test_share_with_video_subattachment():
+    data = {
+        "description": {"text": "Abc"},
+        "media": {
+            "animated_image": None,
+            "image": {
+                "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t15.5256-10/p180x540/1.jpg",
+                "height": 540,
+                "width": 960,
+            },
+            "playable_duration_in_ms": 24469,
+            "is_playable": True,
+            "playable_url": "https://video-arn2-1.xx.fbcdn.net/v/t42.9040-2/vid.mp4",
+        },
+        "source": {"text": "Def"},
+        "style_list": ["attached_story", "fallback"],
+        "title_with_entities": {"text": ""},
+        "properties": [],
+        "url": "https://www.facebook.com/groups/11223344/permalink/1234/",
+        "deduplication_key": "deadbeef123",
+        "action_links": [
+            {"title": None, "url": None},
+            {"title": None, "url": "https://www.facebook.com/groups/11223344/"},
+            {"title": None, "url": None},
+            {"title": "A watch party is currently playing this video.", "url": None},
+        ],
+        "messaging_attribution": None,
+        "messenger_call_to_actions": [],
+        "xma_layout_info": None,
+        "target": {
+            "__typename": "Story",
+            "title": None,
+            "description": {"text": "Abc"},
+            "actors": [
+                {
+                    "__typename": "User",
+                    "name": "Def",
+                    "id": "1111",
+                    "short_name": "Def",
+                    "url": "https://www.facebook.com/some-user",
+                    "profile_picture": {
+                        "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t1.0-1/c1.0.50.50a/p50x50/profile.jpg",
+                        "height": 50,
+                        "width": 50,
+                    },
+                }
+            ],
+            "to": {
+                "__typename": "Group",
+                "name": "Some group",
+                "url": "https://www.facebook.com/groups/11223344/",
+            },
+            "attachments": [
+                {
+                    "url": "https://www.facebook.com/some-user/videos/2222/",
+                    "media": {
+                        "is_playable": True,
+                        "image": {
+                            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t15.5256-10/p180x540/1.jpg",
+                            "height": 540,
+                            "width": 960,
+                        },
+                    },
+                }
+            ],
+            "attached_story": None,
+        },
+        "subattachments": [
+            {
+                "description": None,
+                "media": {
+                    "animated_image": None,
+                    "image": {
+                        "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t15.5256-10/p180x540/1.jpg",
+                        "height": 540,
+                        "width": 960,
+                    },
+                    "playable_duration_in_ms": 24469,
+                    "is_playable": True,
+                    "playable_url": "https://video-arn2-1.xx.fbcdn.net/v/t42.9040-2/vid.mp4",
+                },
+                "source": None,
+                "style_list": [
+                    "video_autoplay",
+                    "video_inline",
+                    "video",
+                    "games_app",
+                    "fallback",
+                ],
+                "title_with_entities": {"text": ""},
+                "properties": [
+                    {
+                        "key": "can_autoplay_result",
+                        "value": {"text": "ugc_default_allowed"},
+                    }
+                ],
+                "url": "https://www.facebook.com/some-user/videos/2222/",
+                "deduplication_key": "deadbeef456",
+                "action_links": [],
+                "messaging_attribution": None,
+                "messenger_call_to_actions": [],
+                "xma_layout_info": None,
+                "target": {
+                    "__typename": "Video",
+                    "video_id": "2222",
+                    "video_messenger_cta_payload": None,
+                },
+            }
+        ],
+    }
+    assert ShareAttachment(
+        author="1111",
+        url="https://www.facebook.com/groups/11223344/permalink/1234/",
+        original_url="https://www.facebook.com/groups/11223344/permalink/1234/",
+        title="",
+        description="Abc",
+        source="Def",
+        image=Image(
+            url="https://scontent-arn2-1.xx.fbcdn.net/v/t15.5256-10/p180x540/1.jpg",
+            width=960,
+            height=540,
+        ),
+        original_image_url="https://scontent-arn2-1.xx.fbcdn.net/v/t15.5256-10/p180x540/1.jpg",
+        attachments=[
+            fbchat.VideoAttachment(
+                id="2222",
+                duration=datetime.timedelta(seconds=24, microseconds=469000),
+                preview_url="https://video-arn2-1.xx.fbcdn.net/v/t42.9040-2/vid.mp4",
+                previews={
+                    Image(
+                        url="https://scontent-arn2-1.xx.fbcdn.net/v/t15.5256-10/p180x540/1.jpg",
+                        width=960,
+                        height=540,
+                    )
+                },
+            )
+        ],
+        id="deadbeef123",
+    ) == ShareAttachment._from_graphql(data)

tests/models/test_file.py

@@ -0,0 +1,358 @@
+import datetime
+import fbchat
+from fbchat import (
+    Image,
+    FileAttachment,
+    AudioAttachment,
+    ImageAttachment,
+    VideoAttachment,
+)
+from fbchat._models._file import graphql_to_attachment, graphql_to_subattachment
+
+
+def test_imageattachment_from_list():
+    data = {
+        "__typename": "MessageImage",
+        "id": "bWVzc2...",
+        "legacy_attachment_id": "1234",
+        "image": {"uri": "https://scontent-arn2-1.xx.fbcdn.net/v/s261x260/1.jpg"},
+        "image1": {
+            "height": 463,
+            "width": 960,
+            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/2.jpg",
+        },
+        "image2": {
+            "height": 988,
+            "width": 2048,
+            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/s2048x2048/3.jpg",
+        },
+        "original_dimensions": {"x": 2833, "y": 1367},
+        "photo_encodings": [],
+    }
+    assert ImageAttachment(
+        id="1234",
+        width=2833,
+        height=1367,
+        previews={
+            Image(url="https://scontent-arn2-1.xx.fbcdn.net/v/s261x260/1.jpg"),
+            Image(
+                url="https://scontent-arn2-1.xx.fbcdn.net/v/2.jpg",
+                width=960,
+                height=463,
+            ),
+            Image(
+                url="https://scontent-arn2-1.xx.fbcdn.net/v/s2048x2048/3.jpg",
+                width=2048,
+                height=988,
+            ),
+        },
+    ) == ImageAttachment._from_list(data)
+
+
+def test_videoattachment_from_list():
+    data = {
+        "__typename": "MessageVideo",
+        "id": "bWVzc2...",
+        "legacy_attachment_id": "1234",
+        "image": {
+            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t15.3394-10/p261x260/1.jpg"
+        },
+        "image1": {
+            "height": 368,
+            "width": 640,
+            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t15.3394-10/2.jpg",
+        },
+        "image2": {
+            "height": 368,
+            "width": 640,
+            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t15.3394-10/3.jpg",
+        },
+        "original_dimensions": {"x": 640, "y": 368},
+    }
+    assert VideoAttachment(
+        id="1234",
+        width=640,
+        height=368,
+        previews={
+            Image(
+                url="https://scontent-arn2-1.xx.fbcdn.net/v/t15.3394-10/p261x260/1.jpg"
+            ),
+            Image(
+                url="https://scontent-arn2-1.xx.fbcdn.net/v/t15.3394-10/2.jpg",
+                width=640,
+                height=368,
+            ),
+            Image(
+                url="https://scontent-arn2-1.xx.fbcdn.net/v/t15.3394-10/3.jpg",
+                width=640,
+                height=368,
+            ),
+        },
+    ) == VideoAttachment._from_list(data)
+
+
+def test_graphql_to_attachment_empty():
+    assert fbchat.Attachment() == graphql_to_attachment({"__typename": "Unknown"})
+
+
+def test_graphql_to_attachment_simple():
+    data = {"__typename": "Unknown", "legacy_attachment_id": "1234"}
+    assert fbchat.Attachment(id="1234") == graphql_to_attachment(data)
+
+
+def test_graphql_to_attachment_file():
+    data = {
+        "__typename": "MessageFile",
+        "attribution_app": None,
+        "attribution_metadata": None,
+        "filename": "file.txt",
+        "url": "https://l.facebook.com/l.php?u=https%3A%2F%2Fcdn.fbsbx.com%2Fv%2Ffile.txt&h=AT1...&s=1",
+        "content_type": "attach:text",
+        "is_malicious": False,
+        "message_file_fbid": "1234",
+        "url_shimhash": "AT0...",
+        "url_skipshim": True,
+    }
+    assert FileAttachment(
+        id="1234",
+        url="https://l.facebook.com/l.php?u=https%3A%2F%2Fcdn.fbsbx.com%2Fv%2Ffile.txt&h=AT1...&s=1",
+        size=None,
+        name="file.txt",
+        is_malicious=False,
+    ) == graphql_to_attachment(data)
+
+
+def test_graphql_to_attachment_audio():
+    data = {
+        "__typename": "MessageAudio",
+        "attribution_app": None,
+        "attribution_metadata": None,
+        "filename": "audio.mp3",
+        "playable_url": "https://cdn.fbsbx.com/v/audio.mp3?dl=1",
+        "playable_duration_in_ms": 27745,
+        "is_voicemail": False,
+        "audio_type": "FILE_ATTACHMENT",
+        "url_shimhash": "AT0...",
+        "url_skipshim": True,
+    }
+    assert AudioAttachment(
+        id=None,
+        filename="audio.mp3",
+        url="https://cdn.fbsbx.com/v/audio.mp3?dl=1",
+        duration=datetime.timedelta(seconds=27, microseconds=745000),
+        audio_type="FILE_ATTACHMENT",
+    ) == graphql_to_attachment(data)
+
+
+def test_graphql_to_attachment_image1():
+    data = {
+        "__typename": "MessageImage",
+        "attribution_app": None,
+        "attribution_metadata": None,
+        "filename": "image-1234",
+        "preview": {
+            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/1.png",
+            "height": 128,
+            "width": 128,
+        },
+        "large_preview": {
+            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/1.png",
+            "height": 128,
+            "width": 128,
+        },
+        "thumbnail": {"uri": "https://scontent-arn2-1.xx.fbcdn.net/v/p50x50/2.png"},
+        "photo_encodings": [],
+        "legacy_attachment_id": "1234",
+        "original_dimensions": {"x": 128, "y": 128},
+        "original_extension": "png",
+        "render_as_sticker": False,
+        "blurred_image_uri": None,
+    }
+    assert ImageAttachment(
+        id="1234",
+        original_extension="png",
+        width=None,
+        height=None,
+        is_animated=False,
+        previews={
+            Image(url="https://scontent-arn2-1.xx.fbcdn.net/v/p50x50/2.png"),
+            Image(
+                url="https://scontent-arn2-1.xx.fbcdn.net/v/1.png",
+                width=128,
+                height=128,
+            ),
+        },
+    ) == graphql_to_attachment(data)
+
+
+def test_graphql_to_attachment_image2():
+    data = {
+        "__typename": "MessageAnimatedImage",
+        "attribution_app": None,
+        "attribution_metadata": None,
+        "filename": "gif-1234",
+        "animated_image": {
+            "uri": "https://cdn.fbsbx.com/v/1.gif",
+            "height": 128,
+            "width": 128,
+        },
+        "legacy_attachment_id": "1234",
+        "preview_image": {
+            "uri": "https://cdn.fbsbx.com/v/1.gif",
+            "height": 128,
+            "width": 128,
+        },
+        "original_dimensions": {"x": 128, "y": 128},
+    }
+    assert ImageAttachment(
+        id="1234",
+        original_extension="gif",
+        width=None,
+        height=None,
+        is_animated=True,
+        previews={Image(url="https://cdn.fbsbx.com/v/1.gif", width=128, height=128)},
+    ) == graphql_to_attachment(data)
+
+
+def test_graphql_to_attachment_video():
+    data = {
+        "__typename": "MessageVideo",
+        "attribution_app": None,
+        "attribution_metadata": None,
+        "filename": "video-4321.mp4",
+        "playable_url": "https://video-arn2-1.xx.fbcdn.net/v/video-4321.mp4",
+        "chat_image": {
+            "height": 96,
+            "width": 168,
+            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/s168x128/1.jpg",
+        },
+        "legacy_attachment_id": "1234",
+        "video_type": "FILE_ATTACHMENT",
+        "original_dimensions": {"x": 640, "y": 368},
+        "playable_duration_in_ms": 6000,
+        "large_image": {
+            "height": 368,
+            "width": 640,
+            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/2.jpg",
+        },
+        "inbox_image": {
+            "height": 260,
+            "width": 452,
+            "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/p261x260/3.jpg",
+        },
+    }
+    assert VideoAttachment(
+        id="1234",
+        width=None,
+        height=None,
+        duration=datetime.timedelta(seconds=6),
+        preview_url="https://video-arn2-1.xx.fbcdn.net/v/video-4321.mp4",
+        previews={
+            Image(
+                url="https://scontent-arn2-1.xx.fbcdn.net/v/s168x128/1.jpg",
+                width=168,
+                height=96,
+            ),
+            Image(
+                url="https://scontent-arn2-1.xx.fbcdn.net/v/p261x260/3.jpg",
+                width=452,
+                height=260,
+            ),
+            Image(
+                url="https://scontent-arn2-1.xx.fbcdn.net/v/2.jpg",
+                width=640,
+                height=368,
+            ),
+        },
+    ) == graphql_to_attachment(data)
+
+
+def test_graphql_to_subattachment_empty():
+    assert None is graphql_to_subattachment({})
+
+
+def test_graphql_to_subattachment_image():
+    data = {
+        "description": {"text": "Abc"},
+        "media": {
+            "animated_image": None,
+            "image": {
+                "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t1.0-9/1.jpg",
+                "height": 960,
+                "width": 720,
+            },
+            "playable_duration_in_ms": 0,
+            "is_playable": False,
+            "playable_url": None,
+        },
+        "source": None,
+        "style_list": ["photo", "games_app", "fallback"],
+        "title_with_entities": {"text": ""},
+        "properties": [
+            {"key": "photoset_reference_token", "value": {"text": "gm.4321"}},
+            {"key": "layout_x", "value": {"text": "0"}},
+            {"key": "layout_y", "value": {"text": "0"}},
+            {"key": "layout_w", "value": {"text": "0"}},
+            {"key": "layout_h", "value": {"text": "0"}},
+        ],
+        "url": "https://www.facebook.com/photo.php?fbid=1234&set=gm.4321&type=3",
+        "deduplication_key": "8334...",
+        "action_links": [],
+        "messaging_attribution": None,
+        "messenger_call_to_actions": [],
+        "xma_layout_info": None,
+        "target": {"__typename": "Photo"},
+    }
+    assert None is graphql_to_subattachment(data)
+
+
+def test_graphql_to_subattachment_video():
+    data = {
+        "description": None,
+        "media": {
+            "animated_image": None,
+            "image": {
+                "uri": "https://scontent-arn2-1.xx.fbcdn.net/v/t15.5256-10/p180x540/1.jpg",
+                "height": 540,
+                "width": 960,
+            },
+            "playable_duration_in_ms": 24469,
+            "is_playable": True,
+            "playable_url": "https://video-arn2-1.xx.fbcdn.net/v/t42.9040-2/vid.mp4",
+        },
+        "source": None,
+        "style_list": [
+            "video_autoplay",
+            "video_inline",
+            "video",
+            "games_app",
+            "fallback",
+        ],
+        "title_with_entities": {"text": ""},
+        "properties": [
+            {"key": "can_autoplay_result", "value": {"text": "ugc_default_allowed"}}
+        ],
+        "url": "https://www.facebook.com/some-username/videos/1234/",
+        "deduplication_key": "ddb7...",
+        "action_links": [],
+        "messaging_attribution": None,
+        "messenger_call_to_actions": [],
+        "xma_layout_info": None,
+        "target": {
+            "__typename": "Video",
+            "video_id": "1234",
+            "video_messenger_cta_payload": None,
+        },
+    }
+    assert VideoAttachment(
+        id="1234",
+        duration=datetime.timedelta(seconds=24, microseconds=469000),
+        preview_url="https://video-arn2-1.xx.fbcdn.net/v/t42.9040-2/vid.mp4",
+        previews={
+            Image(
+                url="https://scontent-arn2-1.xx.fbcdn.net/v/t15.5256-10/p180x540/1.jpg",
+                width=960,
+                height=540,
+            )
+        },
+    ) == graphql_to_subattachment(data)

tests/models/test_location.py

@@ -0,0 +1,96 @@
+import pytest
+import datetime
+import fbchat
+from fbchat import Image, LocationAttachment, LiveLocationAttachment
+
+
+def test_location_attachment_from_graphql():
+    data = {
+        "description": {"text": ""},
+        "media": {
+            "animated_image": None,
+            "image": {
+                "uri": "https://external-arn2-1.xx.fbcdn.net/static_map.php?v=1020&osm_provider=2&size=545x280&zoom=15&markers=55.40000000%2C12.43220000&language=en",
+                "height": 280,
+                "width": 545,
+            },
+            "playable_duration_in_ms": 0,
+            "is_playable": False,
+            "playable_url": None,
+        },
+        "source": None,
+        "style_list": ["message_location", "fallback"],
+        "title_with_entities": {"text": "Your location"},
+        "properties": [
+            {"key": "width", "value": {"text": "545"}},
+            {"key": "height", "value": {"text": "280"}},
+        ],
+        "url": "https://l.facebook.com/l.php?u=https%3A%2F%2Fwww.bing.com%2Fmaps%2Fdefault.aspx%3Fv%3D2%26pc%3DFACEBK%26mid%3D8100%26where1%3D55.4%252C%2B12.4322%26FORM%3DFBKPL1%26mkt%3Den-GB&h=a&s=1",
+        "deduplication_key": "400828513928715",
+        "action_links": [],
+        "messaging_attribution": None,
+        "messenger_call_to_actions": [],
+        "xma_layout_info": None,
+        "target": {"__typename": "MessageLocation"},
+        "subattachments": [],
+    }
+    assert LocationAttachment(
+        id=400828513928715,
+        latitude=55.4,
+        longitude=12.4322,
+        image=Image(
+            url="https://external-arn2-1.xx.fbcdn.net/static_map.php?v=1020&osm_provider=2&size=545x280&zoom=15&markers=55.40000000%2C12.43220000&language=en",
+            width=545,
+            height=280,
+        ),
+        url="https://l.facebook.com/l.php?u=https%3A%2F%2Fwww.bing.com%2Fmaps%2Fdefault.aspx%3Fv%3D2%26pc%3DFACEBK%26mid%3D8100%26where1%3D55.4%252C%2B12.4322%26FORM%3DFBKPL1%26mkt%3Den-GB&h=a&s=1",
+    ) == LocationAttachment._from_graphql(data)
+
+
+@pytest.mark.skip(reason="need to gather test data")
+def test_live_location_from_pull():
+    data = ...
+    assert LiveLocationAttachment(...) == LiveLocationAttachment._from_pull(data)
+
+
+def test_live_location_from_graphql_expired():
+    data = {
+        "description": {"text": "Last update 4 Jan"},
+        "media": None,
+        "source": None,
+        "style_list": ["message_live_location", "fallback"],
+        "title_with_entities": {"text": "Location-sharing ended"},
+        "properties": [],
+        "url": "https://www.facebook.com/",
+        "deduplication_key": "2254535444791641",
+        "action_links": [],
+        "messaging_attribution": None,
+        "messenger_call_to_actions": [],
+        "target": {
+            "__typename": "MessageLiveLocation",
+            "live_location_id": "2254535444791641",
+            "is_expired": True,
+            "expiration_time": 1546626345,
+            "sender": {"id": "100007056224713"},
+            "coordinate": None,
+            "location_title": None,
+            "sender_destination": None,
+            "stop_reason": "CANCELED",
+        },
+        "subattachments": [],
+    }
+    assert LiveLocationAttachment(
+        id=2254535444791641,
+        name="Location-sharing ended",
+        expires_at=datetime.datetime(
+            2019, 1, 4, 18, 25, 45, tzinfo=datetime.timezone.utc
+        ),
+        is_expired=True,
+        url="https://www.facebook.com/",
+    ) == LiveLocationAttachment._from_graphql(data)
+
+
+@pytest.mark.skip(reason="need to gather test data")
+def test_live_location_from_graphql():
+    data = ...
+    assert LiveLocationAttachment(...) == LiveLocationAttachment._from_graphql(data)

tests/models/test_message.py

@@ -0,0 +1,118 @@
+import pytest
+import fbchat
+from fbchat import EmojiSize, Mention, Message, MessageData
+from fbchat._models._message import graphql_to_extensible_attachment
+
+
+@pytest.mark.parametrize(
+    "tags,size",
+    [
+        (None, None),
+        (["hot_emoji_size:unknown"], None),
+        (["bunch", "of:different", "tags:large", "hot_emoji_size:s"], EmojiSize.SMALL),
+        (["hot_emoji_size:s"], EmojiSize.SMALL),
+        (["hot_emoji_size:m"], EmojiSize.MEDIUM),
+        (["hot_emoji_size:l"], EmojiSize.LARGE),
+        (["hot_emoji_size:small"], EmojiSize.SMALL),
+        (["hot_emoji_size:medium"], EmojiSize.MEDIUM),
+        (["hot_emoji_size:large"], EmojiSize.LARGE),
+    ],
+)
+def test_emojisize_from_tags(tags, size):
+    assert size is EmojiSize._from_tags(tags)
+
+
+def test_graphql_to_extensible_attachment_empty():
+    assert None is graphql_to_extensible_attachment({})
+
+
+@pytest.mark.parametrize(
+    "obj,type_",
+    [
+        # UnsentMessage testing is done in test_attachment.py
+        (fbchat.LocationAttachment, "MessageLocation"),
+        (fbchat.LiveLocationAttachment, "MessageLiveLocation"),
+        (fbchat.ShareAttachment, "ExternalUrl"),
+        (fbchat.ShareAttachment, "Story"),
+    ],
+)
+def test_graphql_to_extensible_attachment_dispatch(monkeypatch, obj, type_):
+    monkeypatch.setattr(obj, "_from_graphql", lambda data: True)
+    data = {"story_attachment": {"target": {"__typename": type_}}}
+    assert graphql_to_extensible_attachment(data)
+
+
+def test_mention_from_range():
+    data = {"length": 17, "offset": 0, "entity": {"__typename": "User", "id": "1234"}}
+    assert Mention(thread_id="1234", offset=0, length=17) == Mention._from_range(data)
+    data = {
+        "length": 2,
+        "offset": 10,
+        "entity": {"__typename": "MessengerViewer1To1Thread"},
+    }
+    assert Mention(thread_id=None, offset=10, length=2) == Mention._from_range(data)
+    data = {
+        "length": 5,
+        "offset": 21,
+        "entity": {"__typename": "MessengerViewerGroupThread"},
+    }
+    assert Mention(thread_id=None, offset=21, length=5) == Mention._from_range(data)
+
+
+def test_mention_to_send_data():
+    assert {
+        "profile_xmd[0][id]": "1234",
+        "profile_xmd[0][length]": 7,
+        "profile_xmd[0][offset]": 4,
+        "profile_xmd[0][type]": "p",
+    } == Mention(thread_id="1234", offset=4, length=7)._to_send_data(0)
+    assert {
+        "profile_xmd[1][id]": "4321",
+        "profile_xmd[1][length]": 7,
+        "profile_xmd[1][offset]": 24,
+        "profile_xmd[1][type]": "p",
+    } == Mention(thread_id="4321", offset=24, length=7)._to_send_data(1)
+
+
+def test_message_format_mentions():
+    expected = (
+        "Hey 'Peter'! My name is Michael",
+        [
+            Mention(thread_id="1234", offset=4, length=7),
+            Mention(thread_id="4321", offset=24, length=7),
+        ],
+    )
+    assert expected == Message.format_mentions(
+        "Hey {!r}! My name is {}", ("1234", "Peter"), ("4321", "Michael")
+    )
+    assert expected == Message.format_mentions(
+        "Hey {p!r}! My name is {}", ("4321", "Michael"), p=("1234", "Peter")
+    )
+
+
+def test_message_get_forwarded_from_tags():
+    assert not MessageData._get_forwarded_from_tags(None)
+    assert not MessageData._get_forwarded_from_tags(["hot_emoji_size:unknown"])
+    assert MessageData._get_forwarded_from_tags(
+        ["attachment:photo", "inbox", "sent", "source:chat:forward", "tq"]
+    )
+
+
+@pytest.mark.skip(reason="need to be added")
+def test_message_to_send_data_quick_replies():
+    raise NotImplementedError
+
+
+@pytest.mark.skip(reason="need to gather test data")
+def test_message_from_graphql():
+    pass
+
+
+@pytest.mark.skip(reason="need to gather test data")
+def test_message_from_reply():
+    pass
+
+
+@pytest.mark.skip(reason="need to gather test data")
+def test_message_from_pull():
+    pass

tests/models/test_plan.py

@@ -0,0 +1,155 @@
+import datetime
+from fbchat import GuestStatus, PlanData
+
+
+def test_plan_properties(session):
+    plan = PlanData(
+        session=session,
+        id="1234567890",
+        time=...,
+        title=...,
+        guests={
+            "1234": GuestStatus.INVITED,
+            "2345": GuestStatus.INVITED,
+            "3456": GuestStatus.GOING,
+            "4567": GuestStatus.DECLINED,
+        },
+    )
+    assert set(plan.invited) == {"1234", "2345"}
+    assert plan.going == ["3456"]
+    assert plan.declined == ["4567"]
+
+
+def test_plan_from_pull(session):
+    data = {
+        "event_timezone": "",
+        "event_creator_id": "1234",
+        "event_id": "1111",
+        "event_type": "EVENT",
+        "event_track_rsvp": "1",
+        "event_title": "abc",
+        "event_time": "1500000000",
+        "event_seconds_to_notify_before": "3600",
+        "guest_state_list": (
+            '[{"guest_list_state":"INVITED","node":{"id":"1234"}},'
+            '{"guest_list_state":"INVITED","node":{"id":"2356"}},'
+            '{"guest_list_state":"DECLINED","node":{"id":"3456"}},'
+            '{"guest_list_state":"GOING","node":{"id":"4567"}}]'
+        ),
+    }
+    assert PlanData(
+        session=session,
+        id="1111",
+        time=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+        title="abc",
+        author_id="1234",
+        guests={
+            "1234": GuestStatus.INVITED,
+            "2356": GuestStatus.INVITED,
+            "3456": GuestStatus.DECLINED,
+            "4567": GuestStatus.GOING,
+        },
+    ) == PlanData._from_pull(session, data)
+
+
+def test_plan_from_fetch(session):
+    data = {
+        "message_thread_id": 123456789,
+        "event_time": 1500000000,
+        "creator_id": 1234,
+        "event_time_updated_time": 1450000000,
+        "title": "abc",
+        "track_rsvp": 1,
+        "event_type": "EVENT",
+        "status": "created",
+        "message_id": "mid.xyz",
+        "seconds_to_notify_before": 3600,
+        "event_time_source": "user",
+        "repeat_mode": "once",
+        "creation_time": 1400000000,
+        "location_id": 0,
+        "location_name": None,
+        "latitude": "",
+        "longitude": "",
+        "event_id": 0,
+        "trigger_message_id": "",
+        "note": "",
+        "timezone_id": 0,
+        "end_time": 0,
+        "list_id": 0,
+        "payload_id": 0,
+        "cu_app": "",
+        "location_sharing_subtype": "",
+        "reminder_notif_param": [],
+        "workplace_meeting_id": "",
+        "genie_fbid": 0,
+        "galaxy": "",
+        "oid": 1111,
+        "type": 8128,
+        "is_active": True,
+        "location_address": None,
+        "event_members": {
+            "1234": "INVITED",
+            "2356": "INVITED",
+            "3456": "DECLINED",
+            "4567": "GOING",
+        },
+    }
+    assert PlanData(
+        session=session,
+        id=1111,
+        time=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+        title="abc",
+        location="",
+        location_id="",
+        author_id=1234,
+        guests={
+            "1234": GuestStatus.INVITED,
+            "2356": GuestStatus.INVITED,
+            "3456": GuestStatus.DECLINED,
+            "4567": GuestStatus.GOING,
+        },
+    ) == PlanData._from_fetch(session, data)
+
+
+def test_plan_from_graphql(session):
+    data = {
+        "id": "1111",
+        "lightweight_event_creator": {"id": "1234"},
+        "time": 1500000000,
+        "lightweight_event_type": "EVENT",
+        "location_name": None,
+        "location_coordinates": None,
+        "location_page": None,
+        "lightweight_event_status": "CREATED",
+        "note": "",
+        "repeat_mode": "ONCE",
+        "event_title": "abc",
+        "trigger_message": None,
+        "seconds_to_notify_before": 3600,
+        "allows_rsvp": True,
+        "related_event": None,
+        "event_reminder_members": {
+            "edges": [
+                {"node": {"id": "1234"}, "guest_list_state": "INVITED"},
+                {"node": {"id": "2356"}, "guest_list_state": "INVITED"},
+                {"node": {"id": "3456"}, "guest_list_state": "DECLINED"},
+                {"node": {"id": "4567"}, "guest_list_state": "GOING"},
+            ]
+        },
+    }
+    assert PlanData(
+        session=session,
+        time=datetime.datetime(2017, 7, 14, 2, 40, tzinfo=datetime.timezone.utc),
+        title="abc",
+        location="",
+        location_id="",
+        id="1111",
+        author_id="1234",
+        guests={
+            "1234": GuestStatus.INVITED,
+            "2356": GuestStatus.INVITED,
+            "3456": GuestStatus.DECLINED,
+            "4567": GuestStatus.GOING,
+        },
+    ) == PlanData._from_graphql(session, data)