Version up, thanks to @orenyomtov and @ ThatAlexanderA

* Added `removeFriend` method, #298
* Removed `lxml` from dependencies, #301
* Moved configuration to setup.cfg instead of setup.py

.gitignore

@@ -8,9 +8,11 @@
 # Packages
 *.egg
 *.egg-info
+*.dist-info
 dist
 build
 eggs
+.eggs
 parts
 bin
 var
@@ -24,7 +26,12 @@ develop-eggs
 # Sphinx documentation
 docs/_build/
 
-# Data for tests
+# Scripts and data for tests
+my_tests.py
 my_test_data.json
 my_data.json
-tests.data
+tests.data
+.pytest_cache
+
+# Virtual environment
+venv/

.travis.yml

@@ -0,0 +1,36 @@
+sudo: false
+language: python
+python:
+- 2.7
+- 3.4
+- 3.5
+- 3.6
+- pypy
+
+# There are two accounts made specifically for Travis, and the passwords are really only encrypted for obscurity
+# The global env variables `client1_email`, `client1_password`, `client2_email`, `client2_password` and `group_id` are set on the Travis Settings page
+
+install:
+  - pip install -U -r requirements.txt
+  - pip install -U -r dev-requirements.txt
+
+before_script:
+  - if [[ "$TRAVIS_PYTHON_VERSION" = "2.7" ]]; then export PYTEST_ADDOPTS='-m ""'; fi; # expensive tests (otherwise disabled in pytest.ini)
+  - if [[ "$TRAVIS_PULL_REQUEST" != false ]]; then export PYTEST_ADDOPTS='-m offline'; fi; # offline tests only
+
+script: python -m pytest || python -m pytest --lf; # Run failed tests twice
+
+cache:
+  pip: true
+  directories:
+    - .pytest_cache
+
+deploy:
+  provider: pypi
+  user: madsmtm
+  password:
+    secure: "VA0MLSrwIW/T2KjMwjLZCzrLHw8pJT6tAvb48t7qpBdm8x192hax61pz1TaBZoJvlzyBPFKvluftuclTc7yEFwzXe7Gjqgd/ODKZl/wXDr36hQ7BBOLPZujdwmWLvTzMh3eJZlvkgcLCzrvK3j2oW8cM/+FZeVi/5/FhVuJ4ofs="
+  distributions: sdist bdist_wheel
+  on:
+    branch: master
+    tags: true

MANIFEST.in

@@ -1,4 +1,2 @@
 include LICENSE.txt
-include MANIFEST.in
 include README.rst
-include setup.py

README.rst

@@ -5,21 +5,25 @@ fbchat: Facebook Chat (Messenger) for Python
     :target: LICENSE.txt
     :alt: License: BSD
 
-.. image:: https://img.shields.io/badge/python-2.7%2C%203.4%2C%203.5%2C%203.6-blue.svg
+.. image:: https://img.shields.io/badge/python-2.7%2C%203.4%2C%203.5%2C%203.6%20pypy-blue.svg
     :target: https://pypi.python.org/pypi/fbchat
-    :alt: Supported python versions: 2.7, 3.4, 3.5 and 3.6
+    :alt: Supported python versions: 2.7, 3.4, 3.5, 3.6 and pypy
 
 .. image:: https://readthedocs.org/projects/fbchat/badge/?version=master
     :target: https://fbchat.readthedocs.io
     :alt: Documentation
 
+.. image:: https://travis-ci.org/carpedm20/fbchat.svg?branch=master
+    :target: https://travis-ci.org/carpedm20/fbchat
+    :alt: Travis CI
+
 Facebook Chat (`Messenger <https://www.facebook.com/messages/>`__) for Python.
 This project was inspired by `facebook-chat-api <https://github.com/Schmavery/facebook-chat-api>`__.
 
 **No XMPP or API key is needed**. Just use your email and password.
 
 Go to `Read the Docs <https://fbchat.readthedocs.io>`__ to see the full documentation,
-or jump right into the code by viewing the `examples <examples>`__
+or jump right into the code by viewing the `examples <https://github.com/carpedm20/fbchat/tree/master/examples>`__
 
 Installation:
 
@@ -27,4 +31,17 @@ Installation:
 
     $ pip install fbchat
 
-© Copyright 2015 - 2017 by Taehoon Kim / `@carpedm20 <http://carpedm20.github.io/about/>`__
+You can also install from source, by using `setuptools` (You need at least version 30.3.0):
+
+.. code-block:: console
+
+    $ git clone https://github.com/carpedm20/fbchat.git
+    $ cd fbchat
+    $ python setup.py install
+
+
+Maintainer
+----------
+
+- Mads Marquart / `@madsmtm <https://github.com/madsmtm>`__
+- Taehoon Kim / `@carpedm20 <http://carpedm20.github.io/about/>`__

dev-requirements.txt

@@ -0,0 +1,2 @@
+pytest
+six

docs/intro.rst

@@ -70,8 +70,8 @@ The same method can be applied to some user accounts, though if they've set a cu
 Here's an snippet showing the usage of thread IDs and thread types, where ``<user id>`` and ``<group id>``
 corresponds to the ID of a single user, and the ID of a group respectively::
 
-    client.sendMessage('<message>', thread_id='<user id>', thread_type=ThreadType.USER)
+    client.send(Message(text='<message>'), thread_id='<user id>', thread_type=ThreadType.USER)
-    client.sendMessage('<message>', thread_id='<group id>', thread_type=ThreadType.GROUP)
+    client.send(Message(text='<message>'), thread_id='<group id>', thread_type=ThreadType.GROUP)
 
 Some functions (e.g. :func:`Client.changeThreadColor`) don't require a thread type, so in these cases you just provide the thread ID::
 
@@ -91,7 +91,7 @@ Some of `fbchat`'s functions require these ID's, like :func:`Client.reactToMessa
 and some of then provide this ID, like :func:`Client.sendMessage`.
 This snippet shows how to send a message, and then use the returned ID to react to that message with a 😍 emoji::
 
-    message_id = client.sendMessage('message', thread_id=thread_id, thread_type=thread_type)
+    message_id = client.send(Message(text='message'), thread_id=thread_id, thread_type=thread_type)
     client.reactToMessage(message_id, MessageReaction.LOVE)
 
 
@@ -108,7 +108,7 @@ like adding users to and removing users from a group chat, logically only works
 The simplest way of using `fbchat` is to send a message.
 The following snippet will, as you've probably already figured out, send the message `test message` to your account::
 
-    message_id = client.sendMessage('test message', thread_id=client.uid, thread_type=ThreadType.USER)
+    message_id = client.send(Message(text='test message'), thread_id=client.uid, thread_type=ThreadType.USER)
 
 You can see a full example showing all the possible thread interactions with `fbchat` by going to :ref:`examples`
 
@@ -175,8 +175,8 @@ meaning it will simply print information to the console when an event happens
 The event actions can be changed by subclassing the :class:`Client`, and then overwriting the event methods::
 
     class CustomClient(Client):
-        def onMessage(self, mid, author_id, message, thread_id, thread_type, ts, metadata, msg, **kwargs):
+        def onMessage(self, mid, author_id, message_object, thread_id, thread_type, ts, metadata, msg, **kwargs):
-            # Do something with the message here
+            # Do something with message_object here
             pass
 
     client = CustomClient('<email>', '<password>')
@@ -184,13 +184,13 @@ The event actions can be changed by subclassing the :class:`Client`, and then ov
 **Notice:** The following snippet is as equally valid as the previous one::
 
     class CustomClient(Client):
-        def onMessage(self, message, author_id, thread_id, thread_type, **kwargs):
+        def onMessage(self, message_object, author_id, thread_id, thread_type, **kwargs):
-            # Do something with the message here
+            # Do something with message_object here
             pass
 
     client = CustomClient('<email>', '<password>')
 
-The change was in the parameters that our `onMessage` method took: ``message`` and ``author_id`` got swapped,
+The change was in the parameters that our `onMessage` method took: ``message_object`` and ``author_id`` got swapped,
 and ``mid``, ``ts``, ``metadata`` and ``msg`` got removed, but the function still works, since we included ``**kwargs``
 
 .. note::

examples/basic_usage.py

@@ -7,6 +7,6 @@ client = Client('<email>', '<password>')
 
 print('Own id: {}'.format(client.uid))
 
-client.sendMessage('Hi me!', thread_id=client.uid, thread_type=ThreadType.USER)
+client.send(Message(text='Hi me!'), thread_id=client.uid, thread_type=ThreadType.USER)
 
 client.logout()

examples/echobot.py

@@ -4,15 +4,15 @@ from fbchat import log, Client
 
 # Subclass fbchat.Client and override required methods
 class EchoBot(Client):
-    def onMessage(self, author_id, message, thread_id, thread_type, **kwargs):
+    def onMessage(self, author_id, message_object, thread_id, thread_type, **kwargs):
-        self.markAsDelivered(author_id, thread_id)
+        self.markAsDelivered(thread_id, message_object.uid)
-        self.markAsRead(author_id)
+        self.markAsRead(thread_id)
 
-        log.info("Message from {} in {} ({}): {}".format(author_id, thread_id, thread_type.name, message))
+        log.info("{} from {} in {}".format(message_object, thread_id, thread_type.name))
 
         # If you're not the author, echo
         if author_id != self.uid:
-            self.sendMessage(message, thread_id=thread_id, thread_type=thread_type)
+            self.send(message_object, thread_id=thread_id, thread_type=thread_type)
 
 client = EchoBot("<email>", "<password>")
 client.listen()

examples/interract.py

@@ -9,19 +9,25 @@ thread_id = '1234567890'
 thread_type = ThreadType.GROUP
 
 # Will send a message to the thread
-client.sendMessage('<message>', thread_id=thread_id, thread_type=thread_type)
+client.send(Message(text='<message>'), thread_id=thread_id, thread_type=thread_type)
 
 # Will send the default `like` emoji
-client.sendEmoji(emoji=None, size=EmojiSize.LARGE, thread_id=thread_id, thread_type=thread_type)
+client.send(Message(emoji_size=EmojiSize.LARGE), thread_id=thread_id, thread_type=thread_type)
 
 # Will send the emoji `👍`
-client.sendEmoji(emoji='👍', size=EmojiSize.LARGE, thread_id=thread_id, thread_type=thread_type)
+client.send(Message(text='👍', emoji_size=EmojiSize.LARGE), thread_id=thread_id, thread_type=thread_type)
+
+# Will send the sticker with ID `767334476626295`
+client.send(Message(sticker=Sticker('767334476626295')), thread_id=thread_id, thread_type=thread_type)
+
+# Will send a message with a mention
+client.send(Message(text='This is a @mention', mentions=[Mention(thread_id, offset=10, length=8)]), thread_id=thread_id, thread_type=thread_type)
 
 # Will send the image located at `<image path>`
-client.sendLocalImage('<image path>', message='This is a local image', thread_id=thread_id, thread_type=thread_type)
+client.sendLocalImage('<image path>', message=Message(text='This is a local image'), thread_id=thread_id, thread_type=thread_type)
 
 # Will download the image at the url `<image url>`, and then send it
-client.sendRemoteImage('<image url>', message='This is a remote image', thread_id=thread_id, thread_type=thread_type)
+client.sendRemoteImage('<image url>', message=Message(text='This is a remote image'), thread_id=thread_id, thread_type=thread_type)
 
 
 # Only do these actions if the thread is a group

examples/removebot.py

@@ -4,14 +4,14 @@ from fbchat import log, Client
 from fbchat.models import *
 
 class RemoveBot(Client):
-    def onMessage(self, author_id, message, thread_id, thread_type, **kwargs):
+    def onMessage(self, author_id, message_object, thread_id, thread_type, **kwargs):
         # We can only kick people from group chats, so no need to try if it's a user chat
-        if message == 'Remove me!' and thread_type == ThreadType.GROUP:
+        if message_object.text == 'Remove me!' and thread_type == ThreadType.GROUP:
             log.info('{} will be removed from {}'.format(author_id, thread_id))
             self.removeUserFromGroup(author_id, thread_id=thread_id)
         else:
             # Sends the data to the inherited onMessage, so that we can still see when a message is recieved
-            super(type(self), self).onMessage(author_id=author_id, message=message, thread_id=thread_id, thread_type=thread_type, **kwargs)
+            super(RemoveBot, self).onMessage(author_id=author_id, message_object=message_object, thread_id=thread_id, thread_type=thread_type, **kwargs)
 
 client = RemoveBot("<email>", "<password>")
 client.listen()

fbchat/__init__.py

@@ -1,28 +1,28 @@
 # -*- coding: UTF-8 -*-
 
-from __future__ import unicode_literals
-from datetime import datetime
-from .client import *
-
-
 """
     fbchat
     ~~~~~~
 
     Facebook Chat (Messenger) for Python
 
-    :copyright: (c) 2015 by Taehoon Kim.
+    :copyright: (c) 2015 - 2018 by Taehoon Kim
     :license: BSD, see LICENSE for more details.
 """
 
+from __future__ import unicode_literals
 
-__copyright__ = 'Copyright 2015 - {} by Taehoon Kim'.format(datetime.now().year)
+from .client import *
-__version__ = '1.0.24'
+
+__title__ = 'fbchat'
+__version__ = '1.3.9'
+__description__ = 'Facebook Chat (Messenger) for Python'
+
+__copyright__ = 'Copyright 2015 - 2018 by Taehoon Kim'
 __license__ = 'BSD'
+
 __author__ = 'Taehoon Kim; Moreels Pieter-Jan; Mads Marquart'
 __email__ = 'carpedm20@gmail.com'
-__source__ = 'https://github.com/carpedm20/fbchat/'
-__description__ = 'Facebook Chat (Messenger) for Python'
 
 __all__ = [
     'Client',

fbchat/client.py

@@ -5,14 +5,16 @@ import requests
 import urllib
 from uuid import uuid1
 from random import choice
-from datetime import datetime
 from bs4 import BeautifulSoup as bs
 from mimetypes import guess_type
 from .utils import *
 from .models import *
 from .graphql import *
 import time
-
+try:
+    from urllib.parse import urlparse, parse_qs
+except ImportError:
+    from urlparse import urlparse, parse_qs
 
 
 class Client(object):
@@ -21,6 +23,8 @@ class Client(object):
     See https://fbchat.readthedocs.io for complete documentation of the API.
     """
 
+    ssl_verify = True
+    """Verify ssl certificate, set to False to allow debugging with a proxy"""
     listening = False
     """Whether the client is listening. Used when creating an external event loop to determine when to stop listening"""
     uid = None
@@ -106,7 +110,7 @@ class Client(object):
 
     def _get(self, url, query=None, timeout=30, fix_request=False, as_json=False, error_retries=3):
         payload = self._generatePayload(query)
-        r = self._session.get(url, headers=self._header, params=payload, timeout=timeout)
+        r = self._session.get(url, headers=self._header, params=payload, timeout=timeout, verify=self.ssl_verify)
         if not fix_request:
             return r
         try:
@@ -118,7 +122,7 @@ class Client(object):
 
     def _post(self, url, query=None, timeout=30, fix_request=False, as_json=False, error_retries=3):
         payload = self._generatePayload(query)
-        r = self._session.post(url, headers=self._header, data=payload, timeout=timeout)
+        r = self._session.post(url, headers=self._header, data=payload, timeout=timeout, verify=self.ssl_verify)
         if not fix_request:
             return r
         try:
@@ -137,18 +141,19 @@ class Client(object):
                 return self._graphql(payload, error_retries=error_retries-1)
             raise e
 
-    def _cleanGet(self, url, query=None, timeout=30):
+    def _cleanGet(self, url, query=None, timeout=30, allow_redirects=True):
-        return self._session.get(url, headers=self._header, params=query, timeout=timeout)
+        return self._session.get(url, headers=self._header, params=query, timeout=timeout, verify=self.ssl_verify,
+                                 allow_redirects=allow_redirects)
 
     def _cleanPost(self, url, query=None, timeout=30):
         self.req_counter += 1
-        return self._session.post(url, headers=self._header, data=query, timeout=timeout)
+        return self._session.post(url, headers=self._header, data=query, timeout=timeout, verify=self.ssl_verify)
 
     def _postFile(self, url, files=None, query=None, timeout=30, fix_request=False, as_json=False, error_retries=3):
         payload=self._generatePayload(query)
         # Removes 'Content-Type' from the header
         headers = dict((i, self._header[i]) for i in self._header if i != 'Content-Type')
-        r = self._session.post(url, headers=headers, data=payload, timeout=timeout, files=files)
+        r = self._session.post(url, headers=headers, data=payload, timeout=timeout, files=files, verify=self.ssl_verify)
         if not fix_request:
             return r
         try:
@@ -207,9 +212,19 @@ class Client(object):
         self.ttstamp = ''
 
         r = self._get(self.req_url.BASE)
-        soup = bs(r.text, "lxml")
+        soup = bs(r.text, "html.parser")
-        self.fb_dtsg = soup.find("input", {'name':'fb_dtsg'})['value']
+
-        self.fb_h = soup.find("input", {'name':'h'})['value']
+        fb_dtsg_element = soup.find("input", {'name': 'fb_dtsg'})
+        if fb_dtsg_element:
+            self.fb_dtsg = fb_dtsg_element['value']
+        else:
+            self.fb_dtsg = re.search(r'name="fb_dtsg" value="(.*?)"', r.text).group(1)
+
+
+        fb_h_element = soup.find("input", {'name':'h'})
+        if fb_h_element:
+            self.fb_h = fb_h_element['value']
+
         for i in self.fb_dtsg:
             self.ttstamp += str(ord(i))
         self.ttstamp += '2'
@@ -240,7 +255,7 @@ class Client(object):
         if not (self.email and self.password):
             raise FBchatUserError("Email and password not found.")
 
-        soup = bs(self._get(self.req_url.MOBILE).text, "lxml")
+        soup = bs(self._get(self.req_url.MOBILE).text, "html.parser")
         data = dict((elem['name'], elem['value']) for elem in soup.findAll("input") if elem.has_attr('value') and elem.has_attr('name'))
         data['email'] = self.email
         data['pass'] = self.password
@@ -265,7 +280,7 @@ class Client(object):
             return False, r.url
 
     def _2FA(self, r):
-        soup = bs(r.text, "lxml")
+        soup = bs(r.text, "html.parser")
         data = dict()
 
         s = self.on2FACode()
@@ -324,8 +339,8 @@ class Client(object):
         :rtype: bool
         """
         # Send a request to the login url, to see if we're directed to the home page
-        r = self._cleanGet(self.req_url.LOGIN)
+        r = self._cleanGet(self.req_url.LOGIN, allow_redirects=False)
-        return 'home' in r.url
+        return 'Location' in r.headers and 'home' in r.headers['Location']
 
     def getSession(self):
         """Retrieves session cookies
@@ -399,6 +414,11 @@ class Client(object):
         :return: True if the action was successful
         :rtype: bool
         """
+
+        if not hasattr(self, 'fb_h'):
+            h_r = self._post(self.req_url.MODERN_SETTINGS_MENU, {'pmid': '4'})
+            self.fb_h = re.search(r'name=\\"h\\" value=\\"(.*?)\\"', h_r.text).group(1)
+
         data = {
             'ref': "mb",
             'h': self.fb_h
@@ -480,7 +500,7 @@ class Client(object):
                 if k['id'] in ['0', 0]:
                     # Skip invalid users
                     pass
-                users.append(User(k['id'], first_name=k.get('firstName'), url=k.get('uri'), photo=k.get('thumbSrc'), name=k.get('name'), is_friend=k.get('is_friend'), gender=GENDERS[k.get('gender')]))
+                users.append(User(k['id'], first_name=k.get('firstName'), url=k.get('uri'), photo=k.get('thumbSrc'), name=k.get('name'), is_friend=k.get('is_friend'), gender=GENDERS.get(k.get('gender'))))
 
         return users
 
@@ -513,6 +533,7 @@ class Client(object):
 
         return [graphql_to_page(node) for node in j[name]['pages']['nodes']]
 
+    # TODO intergrate Rooms
     def searchForGroups(self, name, limit=1):
         """
         Find and get group thread by its name
@@ -553,6 +574,7 @@ class Client(object):
             elif node['__typename'] == 'Group':
                 # We don't handle Facebook "Groups"
                 pass
+            # TODO Add Rooms
             else:
                 log.warning('Unknown __typename: {} in {}'.format(repr(node['__typename']), node))
 
@@ -707,6 +729,9 @@ class Client(object):
             if entry.get('thread_type') == 'GROUP':
                 _id = entry['thread_key']['thread_fbid']
                 rtn[_id] = graphql_to_group(entry)
+            elif entry.get('thread_type') == 'ROOM':
+                _id = entry['thread_key']['thread_fbid']
+                rtn[_id] = graphql_to_room(entry)
             elif entry.get('thread_type') == 'ONE_TO_ONE':
                 _id = entry['thread_key']['other_user_id']
                 if pages_and_users.get(_id) is None:
@@ -725,7 +750,7 @@ class Client(object):
         """
         Get the last messages in a thread
 
-        :param thread_id: User/Group ID to default to. See :ref:`intro_threads`
+        :param thread_id: User/Group ID to get messages from. See :ref:`intro_threads`
         :param limit: Max. number of messages to retrieve
         :param before: A timestamp, indicating from which point to retrieve messages
         :type limit: int
@@ -735,6 +760,8 @@ class Client(object):
         :raises: FBchatException if request failed
         """
 
+        thread_id, thread_type = self._getThread(thread_id, None)
+
         j = self.graphql_request(GraphQL(doc_id='1386147188135407', params={
             'id': thread_id,
             'message_limit': limit,
@@ -748,19 +775,23 @@ class Client(object):
 
         return list(reversed([graphql_to_message(message) for message in j['message_thread']['messages']['nodes']]))
 
-    def fetchThreadList(self, offset=0, limit=20, thread_location=ThreadLocation.INBOX):
+    def fetchThreadList(self, offset=None, limit=20, thread_location=ThreadLocation.INBOX, before=None):
         """Get thread list of your facebook account
 
-        :param offset: The offset, from where in the list to recieve threads from
+        :param offset: Deprecated. Do not use!
         :param limit: Max. number of threads to retrieve. Capped at 20
         :param thread_location: models.ThreadLocation: INBOX, PENDING, ARCHIVED or OTHER
-        :type offset: int
+        :param before: A timestamp (in milliseconds), indicating from which point to retrieve threads
         :type limit: int
+        :type before: int
         :return: :class:`models.Thread` objects
         :rtype: list
         :raises: FBchatException if request failed
         """
 
+        if offset is not None:
+            log.warning('Using `offset` in `fetchThreadList` is no longer supported, since Facebook migrated to the use of GraphQL in this request. Use `before` instead')
+
         if limit > 20 or limit < 1:
             raise FBchatUserError('`limit` should be between 1 and 20')
 
@@ -769,61 +800,63 @@ class Client(object):
         else:
             raise FBchatUserError('"thread_location" must be a value of ThreadLocation')
 
-        data = {
+        j = self.graphql_request(GraphQL(doc_id='1349387578499440', params={
-            'client' : self.client,
+            'limit': limit,
-            loc_str + '[offset]' : offset,
+            'tags': [loc_str],
-            loc_str + '[limit]' : limit,
+            'before': before,
-        }
+            'includeDeliveryReceipts': True,
+            'includeSeqID': False
+        }))
 
-        j = self._post(self.req_url.THREADS, data, fix_request=True, as_json=True)
+        return [graphql_to_thread(node) for node in j['viewer']['message_threads']['nodes']]
-        if j.get('payload') is None:
-            raise FBchatException('Missing payload: {}, with data: {}'.format(j, data))
-
-        participants = {}
-        if 'participants' in j['payload']:
-            for p in j['payload']['participants']:
-                if p['type'] == 'page':
-                    participants[p['fbid']] = Page(p['fbid'], url=p['href'], photo=p['image_src'], name=p['name'])
-                elif p['type'] == 'user':
-                    participants[p['fbid']] = User(p['fbid'], url=p['href'], first_name=p['short_name'], is_friend=p['is_friend'], gender=GENDERS[p['gender']], photo=p['image_src'], name=p['name'])
-                else:
-                    raise FBchatException('A participant had an unknown type {}: {}'.format(p['type'], p))
-
-        entries = []
-        if 'threads' in j['payload']:
-            for k in j['payload']['threads']:
-                if k['thread_type'] == 1:
-                    if k['other_user_fbid'] not in participants:
-                        raise FBchatException('The thread {} was not in participants: {}'.format(k, j['payload']))
-                    participants[k['other_user_fbid']].message_count = k['message_count']
-                    entries.append(participants[k['other_user_fbid']])
-                elif k['thread_type'] == 2:
-                    entries.append(Group(k['thread_fbid'], participants=set([p.strip('fbid:') for p in k['participants']]), photo=k['image_src'], name=k['name'], message_count=k['message_count']))
-                else:
-                    raise FBchatException('A thread had an unknown thread type: {}'.format(k))
-
-        return entries
 
     def fetchUnread(self):
         """
-        .. todo::
+        Get the unread thread list
-            Documenting this
 
+        :return: List of unread thread ids
+        :rtype: list
         :raises: FBchatException if request failed
         """
         form = {
-            'client': 'mercury_sync',
             'folders[0]': 'inbox',
+            'client': 'mercury',
             'last_action_timestamp': now() - 60*1000
             # 'last_action_timestamp': 0
         }
 
-        j = self._post(self.req_url.THREAD_SYNC, form, fix_request=True, as_json=True)
+        j = self._post(self.req_url.UNREAD_THREADS, form, fix_request=True, as_json=True)
 
-        return {
+        return j['payload']['unread_thread_fbids'][0]['other_user_fbids']
-            "message_counts": j['payload']['message_counts'],
+
-            "unseen_threads": j['payload']['unseen_thread_ids']
+    def fetchUnseen(self):
-        }
+        """
+        Get the unseen (new) thread list
+
+        :return: List of unseen thread ids
+        :rtype: list
+        :raises: FBchatException if request failed
+        """
+        j = self._post(self.req_url.UNSEEN_THREADS, None, fix_request=True, as_json=True)
+
+        return j['payload']['unseen_thread_fbids'][0]['other_user_fbids']
+
+    def fetchImageUrl(self, image_id):
+        """Fetches the url to the original image from an image attachment ID
+
+        :param image_id: The image you want to fethc
+        :type image_id: str
+        :return: An url where you can download the original image
+        :rtype: str
+        :raises: FBChatException if request failed
+        """
+        image_id = str(image_id)
+        j = check_request(self._get(ReqUrl.ATTACHMENT_PHOTO, query={'photo_id': str(image_id)}))
+
+        url = get_jsmods_require(j, 3)
+        if url is None:
+            raise FBChatException('Could not fetch image url from: {}'.format(j))
+        return url
 
     """
     END FETCH METHODS
@@ -833,45 +866,53 @@ class Client(object):
     SEND METHODS
     """
 
-    def _getSendData(self, thread_id=None, thread_type=ThreadType.USER):
+    def _oldMessage(self, message):
+        return message if isinstance(message, Message) else Message(text=message)
+
+    def _getSendData(self, message=None, thread_id=None, thread_type=ThreadType.USER):
         """Returns the data needed to send a request to `SendURL`"""
         messageAndOTID = generateOfflineThreadingID()
         timestamp = now()
-        date = datetime.now()
         data = {
             'client': self.client,
             'author' : 'fbid:' + str(self.uid),
             'timestamp' : timestamp,
-            'timestamp_absolute' : 'Today',
-            'timestamp_relative' : str(date.hour) + ":" + str(date.minute).zfill(2),
-            'timestamp_time_passed' : '0',
-            'is_unread' : False,
-            'is_cleared' : False,
-            'is_forward' : False,
-            'is_filtered_content' : False,
-            'is_filtered_content_bh': False,
-            'is_filtered_content_account': False,
-            'is_filtered_content_quasar': False,
-            'is_filtered_content_invalid_app': False,
-            'is_spoof_warning' : False,
             'source' : 'source:chat:web',
-            'source_tags[0]' : 'source:chat',
-            'html_body' : False,
-            'ui_push_phase' : 'V3',
-            'status' : '0',
             'offline_threading_id': messageAndOTID,
             'message_id' : messageAndOTID,
             'threading_id': generateMessageID(self.client_id),
-            'ephemeral_ttl_mode:': '0',
+            'ephemeral_ttl_mode:': '0'
-            'manual_retry_cnt' : '0',
-            'signatureID' : getSignatureID()
         }
 
         # Set recipient
         if thread_type in [ThreadType.USER, ThreadType.PAGE]:
-            data["other_user_fbid"] = thread_id
+            data['other_user_fbid'] = thread_id
         elif thread_type == ThreadType.GROUP:
-            data["thread_fbid"] = thread_id
+            data['thread_fbid'] = thread_id
+
+        if message is None:
+            message = Message()
+
+        if message.text or message.sticker or message.emoji_size:
+            data['action_type'] = 'ma-type:user-generated-message'
+
+        if message.text:
+            data['body'] = message.text
+
+        for i, mention in enumerate(message.mentions):
+            data['profile_xmd[{}][id]'.format(i)] = mention.thread_id
+            data['profile_xmd[{}][offset]'.format(i)] = mention.offset
+            data['profile_xmd[{}][length]'.format(i)] = mention.length
+            data['profile_xmd[{}][type]'.format(i)] = 'p'
+
+        if message.emoji_size:
+            if message.text:
+                data['tags[0]'] = 'hot_emoji_size:' + message.emoji_size.name.lower()
+            else:
+                data['sticker_id'] = message.emoji_size.value
+
+        if message.sticker:
+            data['sticker_id'] = message.sticker.uid
 
         return data
 
@@ -888,63 +929,40 @@ class Client(object):
             raise FBchatException('Error when sending message: No message IDs could be found: {}'.format(j))
 
         # update JS token if received in response
-        if j.get('jsmods') is not None and j['jsmods'].get('require') is not None:
+        fb_dtsg = get_jsmods_require(j, 2)
-            try:
+        if fb_dtsg is not None:
-                self.payloadDefault['fb_dtsg'] = j['jsmods']['require'][0][3][0]
+            self.payloadDefault['fb_dtsg'] = fb_dtsg
-            except (KeyError, IndexError) as e:
-                log.warning('Error when updating fb_dtsg. Facebook might have changed protocol!')
 
         return message_id
 
-    def sendMessage(self, message, thread_id=None, thread_type=ThreadType.USER):
+    def send(self, message, thread_id=None, thread_type=ThreadType.USER):
         """
         Sends a message to a thread
 
         :param message: Message to send
         :param thread_id: User/Group ID to send to. See :ref:`intro_threads`
         :param thread_type: See :ref:`intro_threads`
+        :type message: models.Message
         :type thread_type: models.ThreadType
         :return: :ref:`Message ID <intro_message_ids>` of the sent message
         :raises: FBchatException if request failed
         """
         thread_id, thread_type = self._getThread(thread_id, thread_type)
-        data = self._getSendData(thread_id, thread_type)
+        data = self._getSendData(message=message, thread_id=thread_id, thread_type=thread_type)
-
-        data['action_type'] = 'ma-type:user-generated-message'
-        data['body'] = message or ''
-        data['has_attachment'] = False
-        data['specific_to_list[0]'] = 'fbid:' + thread_id
-        data['specific_to_list[1]'] = 'fbid:' + self.uid
 
         return self._doSendRequest(data)
 
+    def sendMessage(self, message, thread_id=None, thread_type=ThreadType.USER):
+        """
+        Deprecated. Use :func:`fbchat.Client.send` instead
+        """
+        return self.send(Message(text=message), thread_id=thread_id, thread_type=thread_type)
+
     def sendEmoji(self, emoji=None, size=EmojiSize.SMALL, thread_id=None, thread_type=ThreadType.USER):
         """
-        Sends an emoji to a thread
+        Deprecated. Use :func:`fbchat.Client.send` instead
-
-        :param emoji: The chosen emoji to send. If not specified, the default `like` emoji is sent
-        :param size: If not specified, a small emoji is sent
-        :param thread_id: User/Group ID to send to. See :ref:`intro_threads`
-        :param thread_type: See :ref:`intro_threads`
-        :type size: models.EmojiSize
-        :type thread_type: models.ThreadType
-        :return: :ref:`Message ID <intro_message_ids>` of the sent emoji
-        :raises: FBchatException if request failed
         """
-        thread_id, thread_type = self._getThread(thread_id, thread_type)
+        return self.send(Message(text=emoji, emoji_size=size), thread_id=thread_id, thread_type=thread_type)
-        data = self._getSendData(thread_id, thread_type)
-        data['action_type'] = 'ma-type:user-generated-message'
-        data['has_attachment'] = False
-        data['specific_to_list[0]'] = 'fbid:' + thread_id
-        data['specific_to_list[1]'] = 'fbid:' + self.uid
-
-        if emoji:
-            data['body'] = emoji
-            data['tags[0]'] = 'hot_emoji_size:' + size.name.lower()
-        else:
-            data["sticker_id"] = size.value
-
-        return self._doSendRequest(data)
 
     def _uploadImage(self, image_path, data, mimetype):
         """Upload an image and get the image_id for sending in a message"""
@@ -964,25 +982,13 @@ class Client(object):
 
     def sendImage(self, image_id, message=None, thread_id=None, thread_type=ThreadType.USER, is_gif=False):
         """
-        Sends an already uploaded image to a thread. (Used by :func:`Client.sendRemoteImage` and :func:`Client.sendLocalImage`)
+        Deprecated. Use :func:`fbchat.Client.send` instead
-
-        :param image_id: ID of an image that's already uploaded to Facebook
-        :param message: Additional message
-        :param thread_id: User/Group ID to send to. See :ref:`intro_threads`
-        :param thread_type: See :ref:`intro_threads`
-        :param is_gif: if sending GIF, True, else False
-        :type thread_type: models.ThreadType
-        :return: :ref:`Message ID <intro_message_ids>` of the sent image
-        :raises: FBchatException if request failed
         """
         thread_id, thread_type = self._getThread(thread_id, thread_type)
-        data = self._getSendData(thread_id, thread_type)
+        data = self._getSendData(message=self._oldMessage(message), thread_id=thread_id, thread_type=thread_type)
 
         data['action_type'] = 'ma-type:user-generated-message'
-        data['body'] = message or ''
         data['has_attachment'] = True
-        data['specific_to_list[0]'] = 'fbid:' + str(thread_id)
-        data['specific_to_list[1]'] = 'fbid:' + str(self.uid)
 
         if not is_gif:
             data['image_ids[0]'] = image_id
@@ -1035,11 +1041,10 @@ class Client(object):
         :param user_ids: One or more user IDs to add
         :param thread_id: Group ID to add people to. See :ref:`intro_threads`
         :type user_ids: list
-        :return: :ref:`Message ID <intro_message_ids>` of the executed action
         :raises: FBchatException if request failed
         """
         thread_id, thread_type = self._getThread(thread_id, None)
-        data = self._getSendData(thread_id, ThreadType.GROUP)
+        data = self._getSendData(thread_id=thread_id, thread_type=ThreadType.GROUP)
 
         data['action_type'] = 'ma-type:log-message'
         data['log_message_type'] = 'log:subscribe'
@@ -1094,7 +1099,7 @@ class Client(object):
             # The thread is a user, so we change the user's nickname
             return self.changeNickname(title, thread_id, thread_id=thread_id, thread_type=thread_type)
         else:
-            data = self._getSendData(thread_id, thread_type)
+            data = self._getSendData(thread_id=thread_id, thread_type=thread_type)
 
             data['action_type'] = 'ma-type:log-message'
             data['log_message_data[name]'] = title
@@ -1135,7 +1140,7 @@ class Client(object):
         thread_id, thread_type = self._getThread(thread_id, None)
 
         data = {
-            'color_choice': color.value,
+            'color_choice': color.value if color != ThreadColor.MESSENGER_BLUE else '',
             'thread_or_other_fbid': thread_id
         }
 
@@ -1192,6 +1197,43 @@ class Client(object):
 
         j = self._post('{}/?{}'.format(self.req_url.MESSAGE_REACTION, url_part), fix_request=True, as_json=True)
 
+    def eventReminder(self, thread_id, time, title, location='', location_id=''):
+        """
+        Sets an event reminder
+
+        ..warning::
+            Does not work in Python2.7
+
+        ..todo::
+            Make this work in Python2.7
+
+        :param thread_id: User/Group ID to send event to. See :ref:`intro_threads`
+        :param time: Event time (unix time stamp)
+        :param title: Event title
+        :param location: Event location name
+        :param location_id: Event location ID
+        :raises: FBchatException if request failed
+        """
+        full_data = {
+            "event_type": "EVENT",
+            "dpr": 1,
+            "event_time" : time,
+            "title" : title,
+            "thread_id" : thread_id,
+            "location_id" : location_id,
+            "location_name" : location,
+            "acontext": {
+                "action_history": [{
+                    "surface": "messenger_chat_tab",
+                    "mechanism": "messenger_composer"
+                }]
+            }
+        }
+        url_part = urllib.parse.urlencode(full_data)
+
+        j = self._post('{}/?{}'.format(self.req_url.EVENT_REMINDER, url_part), fix_request=True, as_json=True)
+
+
     def setTypingStatus(self, status, thread_id=None, thread_type=None):
         """
         Sets users typing status in a thread
@@ -1203,7 +1245,7 @@ class Client(object):
         :type thread_type: models.ThreadType
         :raises: FBchatException if request failed
         """
-        thread_id, thread_type = self._getThread(thread_id, None)
+        thread_id, thread_type = self._getThread(thread_id, thread_type)
 
         data = {
             "typ": status.value,
@@ -1218,28 +1260,36 @@ class Client(object):
     END SEND METHODS
     """
 
-    def markAsDelivered(self, userID, threadID):
+    def markAsDelivered(self, thread_id, message_id):
         """
-        .. todo::
+        Mark a message as delivered
-            Documenting this
+
+        :param thread_id: User/Group ID to which the message belongs. See :ref:`intro_threads`
+        :param message_id: Message ID to set as delivered. See :ref:`intro_threads`
+        :return: Whether the request was successful
+        :raises: FBchatException if request failed
         """
         data = {
-            "message_ids[0]": threadID,
+            "message_ids[0]": message_id,
-            "thread_ids[%s][0]" % userID: threadID
+            "thread_ids[%s][0]" % thread_id: message_id
         }
 
         r = self._post(self.req_url.DELIVERED, data)
         return r.ok
 
-    def markAsRead(self, userID):
+    def markAsRead(self, thread_id):
         """
-        .. todo::
+        Mark a thread as read
-            Documenting this
+        All messages inside the thread will be marked as read
+
+        :param thread_id: User/Group ID to set as read. See :ref:`intro_threads`
+        :return: Whether the request was successful
+        :raises: FBchatException if request failed
         """
         data = {
+            "ids[%s]" % thread_id: 'true',
             "watermarkTimestamp": now(),
-            "shouldSendReadReceipt": True,
+            "shouldSendReadReceipt": 'true',
-            "ids[%s]" % userID: True
         }
 
         r = self._post(self.req_url.READ_STATUS, data)
@@ -1266,6 +1316,25 @@ class Client(object):
         r = self._post(self.req_url.CONNECT, data)
         return r.ok
 
+    def removeFriend(self, friend_id=None):
+        """Removes a specifed friend from your friend list
+
+        :param friend_id: The id of the friend that you want to remove
+        :return: Returns error if the removing was unsuccessful, returns True when successful.
+        """
+        payload = {
+            "friend_id": friend_id,
+            "unref": "none",
+            "confirm": "Confirm",
+        }
+        r = self._post(self.req_url.REMOVE_FRIEND, payload)
+        query = parse_qs(urlparse(r.url).query)
+        if "err" not in query:
+            log.debug("Remove was successful!")
+            return True
+        else:
+            log.warning("Error while removing friend")
+            return False
 
     """
     LISTEN METHODS
@@ -1343,7 +1412,7 @@ class Client(object):
                     delta_type = delta.get("type")
                     metadata = delta.get("messageMetadata")
 
-                    if metadata is not None:
+                    if metadata:
                         mid = metadata["messageId"]
                         author_id = str(metadata['actorFbId'])
                         ts = int(metadata.get("timestamp"))
@@ -1424,9 +1493,52 @@ class Client(object):
 
                     # New message
                     elif delta.get("class") == "NewMessage":
-                        message = delta.get('body', '')
+                        mentions = []
+                        if delta.get('data') and delta['data'].get('prng'):
+                            try:
+                                mentions = [Mention(str(mention.get('i')), offset=mention.get('o'), length=mention.get('l')) for mention in parse_json(delta['data']['prng'])]
+                            except Exception:
+                                log.exception('An exception occured while reading attachments')
+
+                        sticker = None
+                        attachments = []
+                        if delta.get('attachments'):
+                            try:
+                                for a in delta['attachments']:
+                                    mercury = a['mercury']
+                                    if mercury.get('blob_attachment'):
+                                        image_metadata = a.get('imageMetadata', {})
+                                        attach_type = mercury['blob_attachment']['__typename']
+                                        attachment = graphql_to_attachment(mercury.get('blob_attachment', {}))
+
+                                        if attach_type == ['MessageFile', 'MessageVideo', 'MessageAudio']:
+                                            # TODO: Add more data here for audio files
+                                            attachment.size = int(a['fileSize'])
+                                        attachments.append(attachment)
+                                    elif mercury.get('sticker_attachment'):
+                                        sticker = graphql_to_sticker(a['mercury']['sticker_attachment'])
+                                    elif mercury.get('extensible_attachment'):
+                                        # TODO: Add more data here for shared stuff (URLs, events and so on)
+                                        pass
+                            except Exception:
+                                log.exception('An exception occured while reading attachments: {}'.format(delta['attachments']))
+
+                        if metadata and metadata.get('tags'):
+                            emoji_size = get_emojisize_from_tags(metadata.get('tags'))
+
+                        message = Message(
+                            text=delta.get('body'),
+                            mentions=mentions,
+                            emoji_size=emoji_size,
+                            sticker=sticker,
+                            attachments=attachments
+                        )
+                        message.uid = mid
+                        message.author = author_id
+                        message.timestamp = ts
+                        #message.reactions = {}
                         thread_id, thread_type = getThreadIdAndThreadType(metadata)
-                        self.onMessage(mid=mid, author_id=author_id, message=message,
+                        self.onMessage(mid=mid, author_id=author_id, message=delta.get('body', ''), message_object=message,
                                        thread_id=thread_id, thread_type=thread_type, ts=ts, metadata=metadata, msg=m)
 
                     # Unknown message type
@@ -1438,10 +1550,20 @@ class Client(object):
                     self.onInbox(unseen=m["unseen"], unread=m["unread"], recent_unread=m["recent_unread"], msg=m)
 
                 # Typing
-                # elif mtype == "typ":
+                elif mtype == "typ" or mtype == "ttyp":
-                #     author_id = str(m.get("from"))
+                    author_id = str(m.get("from"))
-                #     typing_status = TypingStatus(m.get("st"))
+                    thread_id = m.get("thread_fbid")
-                #     self.onTyping(author_id=author_id, typing_status=typing_status)
+                    if thread_id:
+                        thread_type = ThreadType.GROUP
+                        thread_id = str(thread_id)
+                    else:
+                        thread_type = ThreadType.USER
+                        if author_id == self.uid:
+                            thread_id = m.get("to")
+                        else:
+                            thread_id = author_id
+                    typing_status = TypingStatus(m.get("st"))
+                    self.onTyping(author_id=author_id, status=typing_status, thread_id=thread_id, thread_type=thread_type, msg=m)
 
                 # Delivered
 
@@ -1450,9 +1572,9 @@ class Client(object):
                 #
                 #     self.onSeen(m.get('realtime_viewer_fbid'), m.get('reader'), m.get('time'))
 
-                # elif mtype in ['jewel_requests_add']:
+                elif mtype in ['jewel_requests_add']:
-                #         from_id = m['from']
+                    from_id = m['from']
-                #         self.on_friend_request(from_id)
+                    self.onFriendRequest(from_id=from_id, msg=m)
 
                 # Happens on every login
                 elif mtype == "qprimer":
@@ -1584,23 +1706,25 @@ class Client(object):
         return True
 
 
-    def onMessage(self, mid=None, author_id=None, message=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg={}):
+    def onMessage(self, mid=None, author_id=None, message=None, message_object=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg=None):
         """
         Called when the client is listening, and somebody sends a message
 
         :param mid: The message ID
         :param author_id: The ID of the author
-        :param message: The message
+        :param message: (deprecated. Use `message_object.text` instead)
+        :param message_object: The message (As a `Message` object)
         :param thread_id: Thread ID that the message was sent to. See :ref:`intro_threads`
         :param thread_type: Type of thread that the message was sent to. See :ref:`intro_threads`
         :param ts: The timestamp of the message
         :param metadata: Extra metadata about the message
         :param msg: A full set of the data recieved
+        :type message_object: models.Message
         :type thread_type: models.ThreadType
         """
-        log.info("Message from {} in {} ({}): {}".format(author_id, thread_id, thread_type.name, message))
+        log.info("{} from {} in {}".format(message_object, thread_id, thread_type.name))
 
-    def onColorChange(self, mid=None, author_id=None, new_color=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg={}):
+    def onColorChange(self, mid=None, author_id=None, new_color=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg=None):
         """
         Called when the client is listening, and somebody changes a thread's color
 
@@ -1617,7 +1741,7 @@ class Client(object):
         """
         log.info("Color change from {} in {} ({}): {}".format(author_id, thread_id, thread_type.name, new_color))
 
-    def onEmojiChange(self, mid=None, author_id=None, new_emoji=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg={}):
+    def onEmojiChange(self, mid=None, author_id=None, new_emoji=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg=None):
         """
         Called when the client is listening, and somebody changes a thread's emoji
 
@@ -1633,7 +1757,7 @@ class Client(object):
         """
         log.info("Emoji change from {} in {} ({}): {}".format(author_id, thread_id, thread_type.name, new_emoji))
 
-    def onTitleChange(self, mid=None, author_id=None, new_title=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg={}):
+    def onTitleChange(self, mid=None, author_id=None, new_title=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg=None):
         """
         Called when the client is listening, and somebody changes the title of a thread
 
@@ -1649,7 +1773,7 @@ class Client(object):
         """
         log.info("Title change from {} in {} ({}): {}".format(author_id, thread_id, thread_type.name, new_title))
 
-    def onNicknameChange(self, mid=None, author_id=None, changed_for=None, new_nickname=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg={}):
+    def onNicknameChange(self, mid=None, author_id=None, changed_for=None, new_nickname=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg=None):
         """
         Called when the client is listening, and somebody changes the nickname of a person
 
@@ -1667,7 +1791,7 @@ class Client(object):
         log.info("Nickname change from {} in {} ({}) for {}: {}".format(author_id, thread_id, thread_type.name, changed_for, new_nickname))
 
 
-    def onMessageSeen(self, seen_by=None, thread_id=None, thread_type=ThreadType.USER, seen_ts=None, ts=None, metadata=None, msg={}):
+    def onMessageSeen(self, seen_by=None, thread_id=None, thread_type=ThreadType.USER, seen_ts=None, ts=None, metadata=None, msg=None):
         """
         Called when the client is listening, and somebody marks a message as seen
 
@@ -1682,7 +1806,7 @@ class Client(object):
         """
         log.info("Messages seen by {} in {} ({}) at {}s".format(seen_by, thread_id, thread_type.name, seen_ts/1000))
 
-    def onMessageDelivered(self, msg_ids=None, delivered_for=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg={}):
+    def onMessageDelivered(self, msg_ids=None, delivered_for=None, thread_id=None, thread_type=ThreadType.USER, ts=None, metadata=None, msg=None):
         """
         Called when the client is listening, and somebody marks messages as delivered
 
@@ -1697,7 +1821,7 @@ class Client(object):
         """
         log.info("Messages {} delivered to {} in {} ({}) at {}s".format(msg_ids, delivered_for, thread_id, thread_type.name, ts/1000))
 
-    def onMarkedSeen(self, threads=None, seen_ts=None, ts=None, metadata=None, msg={}):
+    def onMarkedSeen(self, threads=None, seen_ts=None, ts=None, metadata=None, msg=None):
         """
         Called when the client is listening, and the client has successfully marked threads as seen
 
@@ -1712,7 +1836,7 @@ class Client(object):
         log.info("Marked messages as seen in threads {} at {}s".format([(x[0], x[1].name) for x in threads], seen_ts/1000))
 
 
-    def onPeopleAdded(self, mid=None, added_ids=None, author_id=None, thread_id=None, ts=None, msg={}):
+    def onPeopleAdded(self, mid=None, added_ids=None, author_id=None, thread_id=None, ts=None, msg=None):
         """
         Called when the client is listening, and somebody adds people to a group thread
 
@@ -1725,7 +1849,7 @@ class Client(object):
         """
         log.info("{} added: {}".format(author_id, ', '.join(added_ids)))
 
-    def onPersonRemoved(self, mid=None, removed_id=None, author_id=None, thread_id=None, ts=None, msg={}):
+    def onPersonRemoved(self, mid=None, removed_id=None, author_id=None, thread_id=None, ts=None, msg=None):
         """
         Called when the client is listening, and somebody removes a person from a group thread
 
@@ -1738,7 +1862,7 @@ class Client(object):
         """
         log.info("{} removed: {}".format(author_id, removed_id))
 
-    def onFriendRequest(self, from_id=None, msg={}):
+    def onFriendRequest(self, from_id=None, msg=None):
         """
         Called when the client is listening, and somebody sends a friend request
 
@@ -1747,7 +1871,7 @@ class Client(object):
         """
         log.info("Friend request from {}".format(from_id))
 
-    def onInbox(self, unseen=None, unread=None, recent_unread=None, msg={}):
+    def onInbox(self, unseen=None, unread=None, recent_unread=None, msg=None):
         """
         .. todo::
             Documenting this
@@ -1759,7 +1883,21 @@ class Client(object):
         """
         log.info('Inbox event: {}, {}, {}'.format(unseen, unread, recent_unread))
 
-    def onQprimer(self, ts=None, msg={}):
+    def onTyping(self, author_id=None, status=None, thread_id=None, thread_type=None, msg=None):
+        """
+        Called when the client is listening, and somebody starts or stops typing into a chat
+
+        :param author_id: The ID of the person who sent the action
+        :param status: The typing status
+        :param thread_id: Thread ID that the action was sent to. See :ref:`intro_threads`
+        :param thread_type: Type of thread that the action was sent to. See :ref:`intro_threads`
+        :param msg: A full set of the data recieved
+        :type typing_status: models.TypingStatus
+        :type thread_type: models.ThreadType
+        """
+        pass
+
+    def onQprimer(self, ts=None, msg=None):
         """
         Called when the client just started listening
 
@@ -1768,7 +1906,7 @@ class Client(object):
         """
         pass
 
-    def onChatTimestamp(self, buddylist={}, msg={}):
+    def onChatTimestamp(self, buddylist=None, msg=None):
         """
         Called when the client receives chat online presence update
 
@@ -1777,7 +1915,7 @@ class Client(object):
         """
         log.debug('Chat Timestamps received: {}'.format(buddylist))
 
-    def onUnknownMesssageType(self, msg={}):
+    def onUnknownMesssageType(self, msg=None):
         """
         Called when the client is listening, and some unknown data was recieved
 
@@ -1785,7 +1923,7 @@ class Client(object):
         """
         log.debug('Unknown message received: {}'.format(msg))
 
-    def onMessageError(self, exception=None, msg={}):
+    def onMessageError(self, exception=None, msg=None):
         """
         Called when an error was encountered while parsing recieved data
 

fbchat/graphql.py

@@ -42,7 +42,7 @@ def get_customization_info(thread):
         'emoji': info.get('emoji'),
         'color': graphql_color_to_enum(info.get('outgoing_bubble_color'))
     }
-    if thread.get('thread_type') == 'GROUP' or thread.get('is_group_thread') or thread.get('thread_key', {}).get('thread_fbid'):
+    if thread.get('thread_type') in ('GROUP', 'ROOM') or thread.get('is_group_thread') or thread.get('thread_key', {}).get('thread_fbid'):
         rtn['nicknames'] = {}
         for k in info.get('participant_customizations', []):
             rtn['nicknames'][k['participant_id']] = k.get('nickname')
@@ -61,26 +61,95 @@ def get_customization_info(thread):
                 rtn['own_nickname'] = pc[1].get('nickname')
     return rtn
 
+
+def graphql_to_sticker(s):
+    if not s:
+        return None
+    sticker = Sticker(
+        uid=s['id']
+    )
+    if s.get('pack'):
+        sticker.pack = s['pack'].get('id')
+    if s.get('sprite_image'):
+        sticker.is_animated = True
+        sticker.medium_sprite_image = s['sprite_image'].get('uri')
+        sticker.large_sprite_image = s['sprite_image_2x'].get('uri')
+        sticker.frames_per_row = s.get('frames_per_row')
+        sticker.frames_per_col = s.get('frames_per_column')
+        sticker.frame_rate = s.get('frame_rate')
+    sticker.url = s.get('url')
+    sticker.width = s.get('width')
+    sticker.height = s.get('height')
+    if s.get('label'):
+        sticker.label = s['label']
+    return sticker
+
+def graphql_to_attachment(a):
+    _type = a['__typename']
+    if _type in ['MessageImage', 'MessageAnimatedImage']:
+        return ImageAttachment(
+            original_extension=a.get('original_extension') or (a['filename'].split('-')[0] if a.get('filename') else None),
+            width=a.get('original_dimensions', {}).get('width'),
+            height=a.get('original_dimensions', {}).get('height'),
+            is_animated=_type=='MessageAnimatedImage',
+            thumbnail_url=a.get('thumbnail', {}).get('uri'),
+            preview=a.get('preview') or a.get('preview_image'),
+            large_preview=a.get('large_preview'),
+            animated_preview=a.get('animated_image'),
+            uid=a.get('legacy_attachment_id')
+        )
+    elif _type == 'MessageVideo':
+        return VideoAttachment(
+            width=a.get('original_dimensions', {}).get('width'),
+            height=a.get('original_dimensions', {}).get('height'),
+            duration=a.get('playable_duration_in_ms'),
+            preview_url=a.get('playable_url'),
+            small_image=a.get('chat_image'),
+            medium_image=a.get('inbox_image'),
+            large_image=a.get('large_image'),
+            uid=a.get('legacy_attachment_id')
+        )
+    elif _type == 'MessageAudio':
+        return AudioAttachment(
+            filename=a.get('filename'),
+            url=a.get('playable_url'),
+            duration=a.get('playable_duration_in_ms'),
+            audio_type=a.get('audio_type')
+        )
+    elif _type == 'MessageFile':
+        return FileAttachment(
+            url=a.get('url'),
+            name=a.get('filename'),
+            is_malicious=a.get('is_malicious'),
+            uid=a.get('message_file_fbid')
+        )
+    else:
+        return Attachment(
+            uid=a.get('legacy_attachment_id')
+        )
+
 def graphql_to_message(message):
     if message.get('message_sender') is None:
         message['message_sender'] = {}
     if message.get('message') is None:
         message['message'] = {}
-    is_read = None
+    rtn = Message(
-    if message.get('unread') is not None:
-        is_read = not message['unread']
-    return Message(
-        message.get('message_id'),
-        author=message.get('message_sender').get('id'),
-        timestamp=message.get('timestamp_precise'),
-        is_read=is_read,
-        reactions=message.get('message_reactions'),
         text=message.get('message').get('text'),
         mentions=[Mention(m.get('entity', {}).get('id'), offset=m.get('offset'), length=m.get('length')) for m in message.get('message').get('ranges', [])],
-        sticker=message.get('sticker'),
+        emoji_size=get_emojisize_from_tags(message.get('tags_list')),
-        attachments=message.get('blob_attachments'),
+        sticker=graphql_to_sticker(message.get('sticker'))
-        extensible_attachment=message.get('extensible_attachment')
     )
+    rtn.uid = str(message.get('message_id'))
+    rtn.author = str(message.get('message_sender').get('id'))
+    rtn.timestamp = message.get('timestamp_precise')
+    if message.get('unread') is not None:
+        rtn.is_read = not message['unread']
+    rtn.reactions = {str(r['user']['id']):MessageReaction(r['reaction']) for r in message.get('message_reactions')}
+    if message.get('blob_attachments') is not None:
+        rtn.attachments = [graphql_to_attachment(attachment) for attachment in message['blob_attachments']]
+    # TODO: This is still missing parsing:
+    # message.get('extensible_attachment')
+    return rtn
 
 def graphql_to_user(user):
     if user.get('profile_picture') is None:
@@ -92,7 +161,7 @@ def graphql_to_user(user):
         first_name=user.get('first_name'),
         last_name=user.get('last_name'),
         is_friend=user.get('is_viewer_friend'),
-        gender=GENDERS[user.get('gender')],
+        gender=GENDERS.get(user.get('gender')),
         affinity=user.get('affinity'),
         nickname=c_info.get('nickname'),
         color=c_info.get('color'),
@@ -103,10 +172,46 @@ def graphql_to_user(user):
         message_count=user.get('messages_count')
     )
 
+def graphql_to_thread(thread):
+    if thread['thread_type'] == 'GROUP':
+        return graphql_to_group(thread)
+    elif thread['thread_type'] == 'ONE_TO_ONE':
+        if thread.get('big_image_src') is None:
+            thread['big_image_src'] = {}
+        c_info = get_customization_info(thread)
+        participants = [node['messaging_actor'] for node in thread['all_participants']['nodes']]
+        user = next(p for p in participants if p['id'] == thread['thread_key']['other_user_id'])
+        last_message_timestamp = None
+        if 'last_message' in thread:
+            last_message_timestamp = thread['last_message']['nodes'][0]['timestamp_precise']
+
+        return User(
+            user['id'],
+            url=user.get('url'),
+            name=user.get('name'),
+            first_name=user.get('short_name'),
+            last_name=user.get('name').split(user.get('short_name'),1).pop().strip(),
+            is_friend=user.get('is_viewer_friend'),
+            gender=GENDERS.get(user.get('gender')),
+            affinity=user.get('affinity'),
+            nickname=c_info.get('nickname'),
+            color=c_info.get('color'),
+            emoji=c_info.get('emoji'),
+            own_nickname=c_info.get('own_nickname'),
+            photo=user['big_image_src'].get('uri'),
+            message_count=thread.get('messages_count'),
+            last_message_timestamp=last_message_timestamp
+        )
+    else:
+        raise FBchatException('Unknown thread type: {}, with data: {}'.format(thread.get('thread_type'), thread))
+
 def graphql_to_group(group):
     if group.get('image') is None:
         group['image'] = {}
     c_info = get_customization_info(group)
+    last_message_timestamp = None
+    if 'last_message' in group:
+        last_message_timestamp = group['last_message']['nodes'][0]['timestamp_precise']
     return Group(
         group['thread_key']['thread_fbid'],
         participants=set([node['messaging_actor']['id'] for node in group['all_participants']['nodes']]),
@@ -115,7 +220,28 @@ def graphql_to_group(group):
         emoji=c_info.get('emoji'),
         photo=group['image'].get('uri'),
         name=group.get('name'),
-        message_count=group.get('messages_count')
+        message_count=group.get('messages_count'),
+        last_message_timestamp=last_message_timestamp
+    )
+
+def graphql_to_room(room):
+    if room.get('image') is None:
+        room['image'] = {}
+    c_info = get_customization_info(room)
+    return Room(
+        room['thread_key']['thread_fbid'],
+        participants=set([node['messaging_actor']['id'] for node in room['all_participants']['nodes']]),
+        nicknames=c_info.get('nicknames'),
+        color=c_info.get('color'),
+        emoji=c_info.get('emoji'),
+        photo=room['image'].get('uri'),
+        name=room.get('name'),
+        message_count=room.get('messages_count'),
+        admins = set([node.get('id') for node in room.get('thread_admins')]),
+        approval_mode = bool(room.get('approval_mode')),
+        approval_requests = set(node.get('id') for node in room['thread_queue_metadata'].get('approval_requests', {}).get('nodes')),
+        join_link = room['joinable_mode'].get('link'),
+        privacy_mode = bool(room.get('privacy_mode')),
     )
 
 def graphql_to_page(page):
@@ -167,7 +293,9 @@ def graphql_response_to_json(content):
     return rtn
 
 class GraphQL(object):
-    def __init__(self, query=None, doc_id=None, params={}):
+    def __init__(self, query=None, doc_id=None, params=None):
+        if params is None:
+            params = {}
         if query is not None:
             self.value = {
                 'priority': 0,

fbchat/models.py

@@ -9,11 +9,11 @@ class FBchatException(Exception):
 
 class FBchatFacebookError(FBchatException):
     #: The error code that Facebook returned
-    fb_error_code = str
+    fb_error_code = None
     #: The error message that Facebook returned (In the user's own language)
-    fb_error_message = str
+    fb_error_message = None
     #: The status code that was sent in the http response (eg. 404) (Usually only set if not successful, aka. not 200)
-    request_status_code = int
+    request_status_code = None
     def __init__(self, message, fb_error_code=None, fb_error_message=None, request_status_code=None):
         super(FBchatFacebookError, self).__init__(message)
         """Thrown by fbchat when Facebook returns an error"""
@@ -26,17 +26,17 @@ class FBchatUserError(FBchatException):
 
 class Thread(object):
     #: The unique identifier of the thread. Can be used a `thread_id`. See :ref:`intro_threads` for more info
-    uid = str
+    uid = None
     #: Specifies the type of thread. Can be used a `thread_type`. See :ref:`intro_threads` for more info
     type = None
-    #: The thread's picture
+    #: A url to the thread's picture
-    photo = str
+    photo = None
     #: The name of the thread
-    name = str
+    name = None
     #: Timestamp of last message
-    last_message_timestamp = str
+    last_message_timestamp = None
     #: Number of messages in the thread
-    message_count = int
+    message_count = None
     def __init__(self, _type, uid, photo=None, name=None, last_message_timestamp=None, message_count=None):
         """Represents a Facebook thread"""
         self.uid = str(uid)
@@ -55,25 +55,25 @@ class Thread(object):
 
 class User(Thread):
     #: The profile url
-    url = str
+    url = None
     #: The users first name
-    first_name = str
+    first_name = None
     #: The users last name
-    last_name = str
+    last_name = None
     #: Whether the user and the client are friends
-    is_friend = bool
+    is_friend = None
     #: The user's gender
-    gender = str
+    gender = None
     #: From 0 to 1. How close the client is to the user
-    affinity = float
+    affinity = None
     #: The user's nickname
-    nickname = str
+    nickname = None
     #: The clients nickname, as seen by the user
-    own_nickname = str
+    own_nickname = None
     #: A :class:`ThreadColor`. The message color
     color = None
     #: The default emoji
-    emoji = str
+    emoji = None
 
     def __init__(self, uid, url=None, first_name=None, last_name=None, is_friend=None, gender=None, affinity=None, nickname=None, own_nickname=None, color=None, emoji=None, **kwargs):
         """Represents a Facebook user. Inherits `Thread`"""
@@ -92,34 +92,65 @@ class User(Thread):
 
 class Group(Thread):
     #: Unique list (set) of the group thread's participant user IDs
-    participants = set
+    participants = None
-    #: Dict, containing user nicknames mapped to their IDs
+    #: A dict, containing user nicknames mapped to their IDs
-    nicknames = dict
+    nicknames = None
     #: A :class:`ThreadColor`. The groups's message color
     color = None
     #: The groups's default emoji
-    emoji = str
+    emoji = None
 
-    def __init__(self, uid, participants=set(), nicknames=[], color=None, emoji=None, **kwargs):
+    def __init__(self, uid, participants=None, nicknames=None, color=None, emoji=None, **kwargs):
         """Represents a Facebook group. Inherits `Thread`"""
         super(Group, self).__init__(ThreadType.GROUP, uid, **kwargs)
+        if participants is None:
+            participants = set()
         self.participants = participants
+        if nicknames is None:
+            nicknames = []
         self.nicknames = nicknames
         self.color = color
         self.emoji = emoji
 
 
+class Room(Group):
+    # Set containing user IDs of thread admins
+    admins = None
+    # True if users need approval to join
+    approval_mode = None
+    # Set containing user IDs requesting to join
+    approval_requests = None
+    # Link for joining room
+    join_link = None
+    # True is room is not discoverable
+    privacy_mode = None
+
+    def __init__(self, uid, admins=None, approval_mode=None, approval_requests=None, join_link=None, privacy_mode=None, **kwargs):
+        """Represents a Facebook room. Inherits `Group`"""
+        super(Room, self).__init__(uid, **kwargs)
+        self.type = ThreadType.ROOM
+        if admins is None:
+            admins = set()
+        self.admins = admins
+        self.approval_mode = approval_mode
+        if approval_requests is None:
+            approval_requests = set()
+        self.approval_requests = approval_requests
+        self.join_link = join_link
+        self.privacy_mode = privacy_mode
+
+
 class Page(Thread):
     #: The page's custom url
-    url = str
+    url = None
     #: The name of the page's location city
-    city = str
+    city = None
     #: Amount of likes the page has
-    likes = int
+    likes = None
     #: Some extra information about the page
-    sub_title = str
+    sub_title = None
     #: The page's category
-    category = str
+    category = None
 
     def __init__(self, uid, url=None, city=None, likes=None, sub_title=None, category=None, **kwargs):
         """Represents a Facebook page. Inherits `Thread`"""
@@ -132,55 +163,279 @@ class Page(Thread):
 
 
 class Message(object):
-    #: The message ID
-    uid = str
-    #: ID of the sender
-    author = int
-    #: Timestamp of when the message was sent
-    timestamp = str
-    #: Whether the message is read
-    is_read = bool
-    #: A list of message reactions
-    reactions = list
     #: The actual message
-    text = str
+    text = None
     #: A list of :class:`Mention` objects
-    mentions = list
+    mentions = None
-    #: An ID of a sent sticker
+    #: A :class:`EmojiSize`. Size of a sent emoji
-    sticker = str
+    emoji_size = None
+    #: The message ID
+    uid = None
+    #: ID of the sender
+    author = None
+    #: Timestamp of when the message was sent
+    timestamp = None
+    #: Whether the message is read
+    is_read = None
+    #: A dict with user's IDs as keys, and their :class:`MessageReaction` as values
+    reactions = None
+    #: The actual message
+    text = None
+    #: A :class:`Sticker`
+    sticker = None
     #: A list of attachments
-    attachments = list
+    attachments = None
-    #: An extensible attachment, e.g. share object
-    extensible_attachment = dict
 
-    def __init__(self, uid, author=None, timestamp=None, is_read=None, reactions=[], text=None, mentions=[], sticker=None, attachments=[], extensible_attachment={}):
+    def __init__(self, text=None, mentions=None, emoji_size=None, sticker=None, attachments=None):
         """Represents a Facebook message"""
-        self.uid = uid
-        self.author = author
-        self.timestamp = timestamp
-        self.is_read = is_read
-        self.reactions = reactions
         self.text = text
+        if mentions is None:
+            mentions = []
         self.mentions = mentions
+        self.emoji_size = emoji_size
         self.sticker = sticker
+        if attachments is None:
+            attachments = []
         self.attachments = attachments
-        self.extensible_attachment = extensible_attachment
+        self.reactions = {}
+
+    def __repr__(self):
+        return self.__unicode__()
+
+    def __unicode__(self):
+        return '<Message ({}): {}, mentions={} emoji_size={} attachments={}>'.format(self.uid, repr(self.text), self.mentions, self.emoji_size, self.attachments)
+
+class Attachment(object):
+    #: The attachment ID
+    uid = None
+
+    def __init__(self, uid=None):
+        """Represents a Facebook attachment"""
+        self.uid = uid
+
+class Sticker(Attachment):
+    #: The sticker-pack's ID
+    pack = None
+    #: Whether the sticker is animated
+    is_animated = False
+
+    # If the sticker is animated, the following should be present
+    #: URL to a medium spritemap
+    medium_sprite_image = None
+    #: URL to a large spritemap
+    large_sprite_image = None
+    #: The amount of frames present in the spritemap pr. row
+    frames_per_row = None
+    #: The amount of frames present in the spritemap pr. coloumn
+    frames_per_col = None
+    #: The frame rate the spritemap is intended to be played in
+    frame_rate = None
+
+    #: URL to the sticker's image
+    url = None
+    #: Width of the sticker
+    width = None
+    #: Height of the sticker
+    height = None
+    #: The sticker's label/name
+    label = None
+
+    def __init__(self, *args, **kwargs):
+        """Represents a Facebook sticker that has been sent to a Facebook thread as an attachment"""
+        super(Sticker, self).__init__(*args, **kwargs)
+
+class ShareAttachment(Attachment):
+    def __init__(self, **kwargs):
+        """Represents a shared item (eg. URL) that has been sent as a Facebook attachment - *Currently Incomplete!*"""
+        super(ShareAttachment, self).__init__(**kwargs)
+
+class FileAttachment(Attachment):
+    #: Url where you can download the file
+    url = None
+    #: Size of the file in bytes
+    size = None
+    #: Name of the file
+    name = None
+    #: Whether Facebook determines that this file may be harmful
+    is_malicious = None
+
+    def __init__(self, url=None, size=None, name=None, is_malicious=None, **kwargs):
+        """Represents a file that has been sent as a Facebook attachment"""
+        super(FileAttachment, self).__init__(**kwargs)
+        self.url = url
+        self.size = size
+        self.name = name
+        self.is_malicious = is_malicious
+
+class AudioAttachment(Attachment):
+    #: Name of the file
+    filename = None
+    #: Url of the audio file
+    url = None
+    #: Duration of the audioclip in milliseconds
+    duration = None
+    #: Audio type
+    audio_type = None
+
+    def __init__(self, filename=None, url=None, duration=None, audio_type=None, **kwargs):
+        """Represents an audio file that has been sent as a Facebook attachment"""
+        super(AudioAttachment, self).__init__(**kwargs)
+        self.filename = filename
+        self.url = url
+        self.duration = duration
+        self.audio_type = audio_type
+
+class ImageAttachment(Attachment):
+    #: The extension of the original image (eg. 'png')
+    original_extension = None
+    #: Width of original image
+    width = None
+    #: Height of original image
+    height = None
+
+    #: Whether the image is animated
+    is_animated = None
+
+    #: URL to a thumbnail of the image
+    thumbnail_url = None
+
+    #: URL to a medium preview of the image
+    preview_url = None
+    #: Width of the medium preview image
+    preview_width = None
+    #: Height of the medium preview image
+    preview_height = None
+
+    #: URL to a large preview of the image
+    large_preview_url = None
+    #: Width of the large preview image
+    large_preview_width = None
+    #: Height of the large preview image
+    large_preview_height = None
+
+    #: URL to an animated preview of the image (eg. for gifs)
+    animated_preview_url = None
+    #: Width of the animated preview image
+    animated_preview_width = None
+    #: Height of the animated preview image
+    animated_preview_height = None
+
+    def __init__(self, original_extension=None, width=None, height=None, is_animated=None, thumbnail_url=None, preview=None, large_preview=None, animated_preview=None, **kwargs):
+        """
+        Represents an image that has been sent as a Facebook attachment
+        To retrieve the full image url, use: :func:`fbchat.Client.fetchImageUrl`,
+        and pass it the uid of the image attachment
+        """
+        super(ImageAttachment, self).__init__(**kwargs)
+        self.original_extension = original_extension
+        if width is not None:
+            width = int(width)
+        self.width = width
+        if height is not None:
+            height = int(height)
+        self.height = height
+        self.is_animated = is_animated
+        self.thumbnail_url = thumbnail_url
+
+        if preview is None:
+            preview = {}
+        self.preview_url = preview.get('uri')
+        self.preview_width = preview.get('width')
+        self.preview_height = preview.get('height')
+
+        if large_preview is None:
+            large_preview = {}
+        self.large_preview_url = large_preview.get('uri')
+        self.large_preview_width = large_preview.get('width')
+        self.large_preview_height = large_preview.get('height')
+
+        if animated_preview is None:
+            animated_preview = {}
+        self.animated_preview_url = animated_preview.get('uri')
+        self.animated_preview_width = animated_preview.get('width')
+        self.animated_preview_height = animated_preview.get('height')
+
+class VideoAttachment(Attachment):
+    #: Size of the original video in bytes
+    size = None
+    #: Width of original video
+    width = None
+    #: Height of original video
+    height = None
+    #: Length of video in milliseconds
+    duration = None
+    #: URL to very compressed preview video
+    preview_url = None
+
+    #: URL to a small preview image of the video
+    small_image_url = None
+    #: Width of the small preview image
+    small_image_width = None
+    #: Height of the small preview image
+    small_image_height = None
+
+    #: URL to a medium preview image of the video
+    medium_image_url = None
+    #: Width of the medium preview image
+    medium_image_width = None
+    #: Height of the medium preview image
+    medium_image_height = None
+
+    #: URL to a large preview image of the video
+    large_image_url = None
+    #: Width of the large preview image
+    large_image_width = None
+    #: Height of the large preview image
+    large_image_height = None
+
+    def __init__(self, size=None, width=None, height=None, duration=None, preview_url=None, small_image=None, medium_image=None, large_image=None, **kwargs):
+        """Represents a video that has been sent as a Facebook attachment"""
+        super(VideoAttachment, self).__init__(**kwargs)
+        self.size = size
+        self.width = width
+        self.height = height
+        self.duration = duration
+        self.preview_url = preview_url
+
+        if small_image is None:
+            small_image = {}
+        self.small_image_url = small_image.get('uri')
+        self.small_image_width = small_image.get('width')
+        self.small_image_height = small_image.get('height')
+
+        if medium_image is None:
+            medium_image = {}
+        self.medium_image_url = medium_image.get('uri')
+        self.medium_image_width = medium_image.get('width')
+        self.medium_image_height = medium_image.get('height')
+
+        if large_image is None:
+            large_image = {}
+        self.large_image_url = large_image.get('uri')
+        self.large_image_width = large_image.get('width')
+        self.large_image_height = large_image.get('height')
 
 
 class Mention(object):
-    #: The user ID the mention is pointing at
+    #: The thread ID the mention is pointing at
-    user_id = str
+    thread_id = None
     #: The character where the mention starts
-    offset = int
+    offset = None
     #: The length of the mention
-    length = int
+    length = None
 
-    def __init__(self, user_id, offset=0, length=10):
+    def __init__(self, thread_id, offset=0, length=10):
         """Represents a @mention"""
-        self.user_id = user_id
+        self.thread_id = thread_id
         self.offset = offset
         self.length = length
 
+    def __repr__(self):
+        return self.__unicode__()
+
+    def __unicode__(self):
+        return '<Mention {}: offset={} length={}>'.format(self.thread_id, self.offset, self.length)
+
 class Enum(enum.Enum):
     """Used internally by fbchat to support enumerations"""
     def __repr__(self):
@@ -192,13 +447,14 @@ class ThreadType(Enum):
     USER = 1
     GROUP = 2
     PAGE = 3
+    ROOM = 4
 
 class ThreadLocation(Enum):
     """Used to specify where a thread is located (inbox, pending, archived, other)."""
-    INBOX = 'inbox'
+    INBOX = 'INBOX'
-    PENDING = 'pending'
+    PENDING = 'PENDING'
-    ARCHIVED = 'action:archived'
+    ARCHIVED = 'ARCHIVED'
-    OTHER = 'other'
+    OTHER = 'OTHER'
 
 class TypingStatus(Enum):
     """Used to specify whether the user is typing or has stopped typing"""
@@ -213,7 +469,7 @@ class EmojiSize(Enum):
 
 class ThreadColor(Enum):
     """Used to specify a thread colors"""
-    MESSENGER_BLUE = ''
+    MESSENGER_BLUE = '#0084ff'
     VIKING = '#44bec7'
     GOLDEN_POPPY = '#ffc300'
     RADICAL_RED = '#fa3c4c'
@@ -238,22 +494,3 @@ class MessageReaction(Enum):
     ANGRY = '😠'
     YES = '👍'
     NO = '👎'
-
-LIKES = {
-    'large': EmojiSize.LARGE,
-    'medium': EmojiSize.MEDIUM,
-    'small': EmojiSize.SMALL,
-    'l': EmojiSize.LARGE,
-    'm': EmojiSize.MEDIUM,
-    's': EmojiSize.SMALL
-}
-
-MessageReactionFix = {
-    '😍': ('0001f60d', '%F0%9F%98%8D'),
-    '😆': ('0001f606', '%F0%9F%98%86'),
-    '😮': ('0001f62e', '%F0%9F%98%AE'),
-    '😢': ('0001f622', '%F0%9F%98%A2'),
-    '😠': ('0001f620', '%F0%9F%98%A0'),
-    '👍': ('0001f44d', '%F0%9F%91%8D'),
-    '👎': ('0001f44e', '%F0%9F%91%8E')
-}

fbchat/utils.py

@@ -9,6 +9,13 @@ import warnings
 import logging
 from .models import *
 
+try:
+    from urllib.parse import urlencode
+    basestring = (str, bytes)
+except ImportError:
+    from urllib import urlencode
+    basestring = basestring
+
 # Python 2's `input` executes the input, whereas `raw_input` just returns the input
 try:
     input = raw_input
@@ -32,6 +39,26 @@ USER_AGENTS = [
     "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.0 Safari/536.6"
 ]
 
+LIKES = {
+    'large': EmojiSize.LARGE,
+    'medium': EmojiSize.MEDIUM,
+    'small': EmojiSize.SMALL,
+    'l': EmojiSize.LARGE,
+    'm': EmojiSize.MEDIUM,
+    's': EmojiSize.SMALL
+}
+
+MessageReactionFix = {
+    '😍': ('0001f60d', '%F0%9F%98%8D'),
+    '😆': ('0001f606', '%F0%9F%98%86'),
+    '😮': ('0001f62e', '%F0%9F%98%AE'),
+    '😢': ('0001f622', '%F0%9F%98%A2'),
+    '😠': ('0001f620', '%F0%9F%98%A0'),
+    '👍': ('0001f44d', '%F0%9F%91%8D'),
+    '👎': ('0001f44e', '%F0%9F%91%8E')
+}
+
+
 GENDERS = {
     # For standard requests
     0: 'unknown',
@@ -54,13 +81,12 @@ GENDERS = {
     #'': 'female_singular_guess',
     #'': 'male_singular_guess',
     #'': 'mixed',
-    #'': 'neuter_singular',
+    'NEUTER': 'neuter_singular',
     #'': 'unknown_singular',
     #'': 'female_plural',
     #'': 'male_plural',
     #'': 'neuter_plural',
     #'': 'unknown_plural',
-    None: None
 }
 
 class ReqUrl(object):
@@ -68,7 +94,8 @@ class ReqUrl(object):
     SEARCH = "https://www.facebook.com/ajax/typeahead/search.php"
     LOGIN = "https://m.facebook.com/login.php?login_attempt=1"
     SEND = "https://www.facebook.com/messaging/send/"
-    THREAD_SYNC = "https://www.facebook.com/ajax/mercury/thread_sync.php"
+    UNREAD_THREADS = "https://www.facebook.com/ajax/mercury/unread_threads.php"
+    UNSEEN_THREADS = "https://www.facebook.com/mercury/unseen_thread_ids/"
     THREADS = "https://www.facebook.com/ajax/mercury/threadlist_info.php"
     MESSAGES = "https://www.facebook.com/ajax/mercury/thread_info.php"
     READ_STATUS = "https://www.facebook.com/ajax/mercury/change_read_status.php"
@@ -92,7 +119,11 @@ class ReqUrl(object):
     MESSAGE_REACTION = "https://www.facebook.com/webgraphql/mutation"
     TYPING = "https://www.facebook.com/ajax/messaging/typ.php"
     GRAPHQL = "https://www.facebook.com/api/graphqlbatch/"
-
+    ATTACHMENT_PHOTO = "https://www.facebook.com/mercury/attachments/photo/"
+    EVENT_REMINDER = "https://www.facebook.com/ajax/eventreminder/create"
+    MODERN_SETTINGS_MENU = "https://www.facebook.com/bluebar/modern_settings_menu/"
+    REMOVE_FRIEND = "https://m.facebook.com/a/removefriend.php"
+    
     pull_channel = 0
 
     def change_pull_channel(self, channel=None):
@@ -121,6 +152,9 @@ def get_decoded_r(r):
 def get_decoded(content):
     return content.decode(facebookEncoding)
 
+def parse_json(content):
+    return json.loads(content)
+
 def get_json(r):
     return json.loads(strip_to_json(get_decoded_r(r)))
 
@@ -182,3 +216,22 @@ def check_request(r, as_json=True):
         return j
     else:
         return content
+
+def get_jsmods_require(j, index):
+    if j.get('jsmods') and j['jsmods'].get('require'):
+        try:
+            return j['jsmods']['require'][0][index][0]
+        except (KeyError, IndexError) as e:
+            log.warning('Error when getting jsmods_require: {}. Facebook might have changed protocol'.format(j))
+    return None
+
+def get_emojisize_from_tags(tags):
+    if tags is None:
+        return None
+    tmp = [tag for tag in tags if tag.startswith('hot_emoji_size:')]
+    if len(tmp) > 0:
+        try:
+            return LIKES[tmp[0].split(':')[1]]
+        except (KeyError, IndexError):
+            log.exception('Could not determine emoji size from {} - {}'.format(tags, tmp))
+    return None

pytest.ini

@@ -0,0 +1,6 @@
+[pytest]
+xfail_strict=true
+markers =
+    offline: Offline tests, aka. tests that can be executed without the need of a client
+    expensive: Expensive tests, which should be executed sparingly
+addopts = -m "not expensive"

requirements.txt

@@ -1,4 +1,3 @@
 requests
-lxml
 beautifulsoup4
-enum34; python_version == '2.7'
+enum34; python_version < '3.4'

setup.cfg

@@ -0,0 +1,51 @@
+[metadata]
+name = fbchat
+version = attr: fbchat.__version__
+license = BSD
+license_file = LICENSE.txt
+
+author = Taehoon Kim
+author_email = carpedm20@gmail.com
+maintainer = Mads Marquart
+maintainer_email = madsmtm@gmail.com
+
+description = Facebook Chat (Messenger) for Python
+long_description = file: README.rst
+long_description_content_type = text/x-rst
+
+keywords = Facebook FB Messenger Chat Api Bot
+classifiers =
+    Development Status :: 3 - Alpha
+    Intended Audience :: Developers
+    Intended Audience :: Information Technology
+    License :: OSI Approved :: BSD License
+    Operating System :: OS Independent
+    Natural Language :: English
+    Programming Language :: Python
+    Programming Language :: Python :: 2.7
+    Programming Language :: Python :: 3.4
+    Programming Language :: Python :: 3.5
+    Programming Language :: Python :: 3.6
+    Programming Language :: Python :: Implementation :: CPython
+    Programming Language :: Python :: Implementation :: PyPy
+    Topic :: Communications :: Chat
+    Topic :: Internet :: WWW/HTTP :: Dynamic Content
+    Topic :: Software Development :: Libraries :: Python Modules
+
+url = https://github.com/carpedm20/fbchat/
+project_urls =
+    Documentation = https://fbchat.readthedocs.io/
+    Repository = https://github.com/carpedm20/fbchat/
+
+[options]
+zip_safe = True
+include_package_data = True
+packages = find:
+python_requires = >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4.0
+install_requires =
+    requests
+    beautifulsoup4
+    # May not work in pip with bdist_wheel
+    # See https://wheel.readthedocs.io/en/latest/#defining-conditional-dependencies
+    # It is therefore defined in setup.py
+    # enum34; python_version < '3.4'

setup.py

@@ -1,83 +1,8 @@
 #!/usr/bin/env python
+# -*- coding: UTF-8 -*-
 
+from __future__ import unicode_literals
 
-"""
+from setuptools import setup
-Setup script for fbchat
-"""
 
-
+setup(extras_require={':python_version < "3.4"': ['enum34']})
-import os
-try:
-    from setuptools import setup
-except ImportError:
-    from distutils.core import setup
-
-
-with open('README.rst') as f:
-    readme_content = f.read().strip()
-
-requirements = [
-    'requests',
-    'lxml',
-    'beautifulsoup4',
-    "enum34; python_version == '2.7'"
-]
-
-version = None
-author = None
-email = None
-source = None
-description = None
-with open(os.path.join('fbchat', '__init__.py')) as f:
-    for line in f:
-        if line.strip().startswith('__version__'):
-            version = line.split('=')[1].strip().replace('"', '').replace("'", '')
-        elif line.strip().startswith('__author__'):
-            author = line.split('=')[1].strip().replace('"', '').replace("'", '')
-        elif line.strip().startswith('__email__'):
-            email = line.split('=')[1].strip().replace('"', '').replace("'", '')
-        elif line.strip().startswith('__source__'):
-            source = line.split('=')[1].strip().replace('"', '').replace("'", '')
-        elif line.strip().startswith('__description__'):
-            description = line.split('=')[1].strip().replace('"', '').replace("'", '')
-        elif None not in (version, author, email, source, description):
-            break
-
-setup(
-    name='fbchat',
-    author=author,
-    author_email=email,
-    license='BSD License',
-    keywords=["facebook chat fbchat"],
-    description=description,
-    long_description=readme_content,
-    classifiers=[
-        'Development Status :: 2 - Pre-Alpha',
-        'Intended Audience :: Developers',
-        'Intended Audience :: Developers',
-        'Intended Audience :: Information Technology',
-        'License :: OSI Approved :: BSD License',
-        'License :: OSI Approved :: BSD License',
-        'Operating System :: OS Independent',
-        'Programming Language :: Python :: 2.6',
-        'Programming Language :: Python :: 2.7',
-        'Programming Language :: Python :: 2.7',
-        'Programming Language :: Python :: 3.2',
-        'Programming Language :: Python :: 3.3',
-        'Programming Language :: Python :: 3.3',
-        'Programming Language :: Python :: 3.4',
-        'Programming Language :: Python :: 3.5',
-        'Programming Language :: Python :: Implementation :: CPython',
-        'Programming Language :: Python :: Implementation :: PyPy',
-        'Programming Language :: Python',
-        'Topic :: Software Development :: Libraries :: Python Modules',
-        'Topic :: Internet :: WWW/HTTP :: Dynamic Content',
-        'Topic :: Communications :: Chat',
-    ],
-    include_package_data=True,
-    packages=['fbchat'],
-    install_requires=requirements,
-    url=source,
-    version=version,
-    zip_safe=True,
-)

tests.py

@@ -1,238 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: UTF-8 -*-
-
-from __future__ import unicode_literals
-import json
-import logging
-import unittest
-from getpass import getpass
-from sys import argv
-from os import path, chdir
-from glob import glob
-from fbchat import Client
-from fbchat.models import *
-import py_compile
-
-logging_level = logging.ERROR
-
-"""
-
-Testing script for `fbchat`.
-Full documentation on https://fbchat.readthedocs.io/
-
-"""
-
-class CustomClient(Client):
-    def __init__(self, *args, **kwargs):
-        self.got_qprimer = False
-        super(type(self), self).__init__(*args, **kwargs)
-
-    def onQprimer(self, msg, **kwargs):
-        self.got_qprimer = True
-
-class TestFbchat(unittest.TestCase):
-    def test_examples(self):
-        # Checks for syntax errors in the examples
-        chdir('examples')
-        for f in glob('*.txt'):
-            print(f)
-            with self.assertRaises(py_compile.PyCompileError):
-                py_compile.compile(f)
-
-        chdir('..')
-
-    def test_loginFunctions(self):
-        self.assertTrue(client.isLoggedIn())
-
-        client.logout()
-
-        self.assertFalse(client.isLoggedIn())
-
-        with self.assertRaises(Exception):
-            client.login('<email>', '<password>', max_tries=1)
-
-        client.login(email, password)
-
-        self.assertTrue(client.isLoggedIn())
-
-    def test_sessions(self):
-        global client
-        session_cookies = client.getSession()
-        client = CustomClient(email, password, session_cookies=session_cookies, logging_level=logging_level)
-
-        self.assertTrue(client.isLoggedIn())
-
-    def test_defaultThread(self):
-        # setDefaultThread
-        client.setDefaultThread(group_id, ThreadType.GROUP)
-        self.assertTrue(client.sendMessage('test_default_recipient★'))
-
-        client.setDefaultThread(user_id, ThreadType.USER)
-        self.assertTrue(client.sendMessage('test_default_recipient★'))
-
-        # resetDefaultThread
-        client.resetDefaultThread()
-        with self.assertRaises(ValueError):
-            client.sendMessage('should_not_send')
-
-    def test_fetchAllUsers(self):
-        users = client.fetchAllUsers()
-        self.assertGreater(len(users), 0)
-
-    def test_searchFor(self):
-        users = client.searchForUsers('Mark Zuckerberg')
-        self.assertGreater(len(users), 0)
-
-        u = users[0]
-
-        # Test if values are set correctly
-        self.assertEqual(u.uid, '4')
-        self.assertEqual(u.type, ThreadType.USER)
-        self.assertEqual(u.photo[:4], 'http')
-        self.assertEqual(u.url[:4], 'http')
-        self.assertEqual(u.name, 'Mark Zuckerberg')
-
-        group_name = client.changeThreadTitle('tést_searchFor', thread_id=group_id, thread_type=ThreadType.GROUP)
-        groups = client.searchForGroups('té')
-        self.assertGreater(len(groups), 0)
-
-    def test_sendEmoji(self):
-        self.assertIsNotNone(client.sendEmoji(size=EmojiSize.SMALL, thread_id=user_id, thread_type=ThreadType.USER))
-        self.assertIsNotNone(client.sendEmoji(size=EmojiSize.MEDIUM, thread_id=user_id, thread_type=ThreadType.USER))
-        self.assertIsNotNone(client.sendEmoji('😆', EmojiSize.LARGE, user_id, ThreadType.USER))
-
-        self.assertIsNotNone(client.sendEmoji(size=EmojiSize.SMALL, thread_id=group_id, thread_type=ThreadType.GROUP))
-        self.assertIsNotNone(client.sendEmoji(size=EmojiSize.MEDIUM, thread_id=group_id, thread_type=ThreadType.GROUP))
-        self.assertIsNotNone(client.sendEmoji('😆', EmojiSize.LARGE, group_id, ThreadType.GROUP))
-
-    def test_sendMessage(self):
-        self.assertIsNotNone(client.sendMessage('test_send_user★', user_id, ThreadType.USER))
-        self.assertIsNotNone(client.sendMessage('test_send_group★', group_id, ThreadType.GROUP))
-        with self.assertRaises(Exception):
-            client.sendMessage('test_send_user_should_fail★', user_id, ThreadType.GROUP)
-        with self.assertRaises(Exception):
-            client.sendMessage('test_send_group_should_fail★', group_id, ThreadType.USER)
-
-    def test_sendImages(self):
-        image_url = 'https://cdn4.iconfinder.com/data/icons/ionicons/512/icon-image-128.png'
-        image_local_url = path.join(path.dirname(__file__), 'tests/image.png')
-        self.assertTrue(client.sendRemoteImage(image_url, 'test_send_user_images_remote★', user_id, ThreadType.USER))
-        self.assertTrue(client.sendRemoteImage(image_url, 'test_send_group_images_remote★', group_id, ThreadType.GROUP))
-        self.assertTrue(client.sendLocalImage(image_local_url, 'test_send_group_images_local★', user_id, ThreadType.USER))
-        self.assertTrue(client.sendLocalImage(image_local_url, 'test_send_group_images_local★', group_id, ThreadType.GROUP))
-
-    def test_fetchThreadList(self):
-        client.fetchThreadList(offset=0, limit=20)
-
-    def test_fetchThreadMessages(self):
-        client.sendMessage('test_user_getThreadInfo★', thread_id=user_id, thread_type=ThreadType.USER)
-
-        messages = client.fetchThreadMessages(thread_id=user_id, limit=1)
-        self.assertEqual(messages[0].author, client.uid)
-        self.assertEqual(messages[0].text, 'test_user_getThreadInfo★')
-
-        client.sendMessage('test_group_getThreadInfo★', thread_id=group_id, thread_type=ThreadType.GROUP)
-
-        messages = client.fetchThreadMessages(thread_id=group_id, limit=1)
-        self.assertEqual(messages[0].author, client.uid)
-        self.assertEqual(messages[0].text, 'test_group_getThreadInfo★')
-
-    def test_listen(self):
-        client.startListening()
-        client.doOneListen()
-        client.stopListening()
-
-        self.assertTrue(client.got_qprimer)
-
-    def test_fetchInfo(self):
-        info = client.fetchUserInfo('4')['4']
-        self.assertEqual(info.name, 'Mark Zuckerberg')
-
-        info = client.fetchGroupInfo(group_id)[group_id]
-        self.assertEqual(info.type, ThreadType.GROUP)
-
-    def test_removeAddFromGroup(self):
-        client.removeUserFromGroup(user_id, thread_id=group_id)
-        client.addUsersToGroup(user_id, thread_id=group_id)
-
-    def test_changeThreadTitle(self):
-        client.changeThreadTitle('test_changeThreadTitle★', thread_id=group_id, thread_type=ThreadType.GROUP)
-        client.changeThreadTitle('test_changeThreadTitle★', thread_id=user_id, thread_type=ThreadType.USER)
-
-    def test_changeNickname(self):
-        client.changeNickname('test_changeNicknameSelf★', client.uid, thread_id=user_id, thread_type=ThreadType.USER)
-        client.changeNickname('test_changeNicknameOther★', user_id, thread_id=user_id, thread_type=ThreadType.USER)
-        client.changeNickname('test_changeNicknameSelf★', client.uid, thread_id=group_id, thread_type=ThreadType.GROUP)
-        client.changeNickname('test_changeNicknameOther★', user_id, thread_id=group_id, thread_type=ThreadType.GROUP)
-
-    def test_changeThreadEmoji(self):
-        client.changeThreadEmoji('😀', group_id)
-        client.changeThreadEmoji('😀', user_id)
-        client.changeThreadEmoji('😆', group_id)
-        client.changeThreadEmoji('😆', user_id)
-
-    def test_changeThreadColor(self):
-        client.changeThreadColor(ThreadColor.BRILLIANT_ROSE, group_id)
-        client.changeThreadColor(ThreadColor.MESSENGER_BLUE, group_id)
-        client.changeThreadColor(ThreadColor.BRILLIANT_ROSE, user_id)
-        client.changeThreadColor(ThreadColor.MESSENGER_BLUE, user_id)
-
-    def test_reactToMessage(self):
-        mid = client.sendMessage('test_reactToMessage★', user_id, ThreadType.USER)
-        client.reactToMessage(mid, MessageReaction.LOVE)
-        mid = client.sendMessage('test_reactToMessage★', group_id, ThreadType.GROUP)
-        client.reactToMessage(mid, MessageReaction.LOVE)
-
-    def test_setTypingStatus(self):
-        client.setTypingStatus(TypingStatus.TYPING, thread_id=user_id, thread_type=ThreadType.USER)
-        client.setTypingStatus(TypingStatus.STOPPED, thread_id=user_id, thread_type=ThreadType.USER)
-        client.setTypingStatus(TypingStatus.TYPING, thread_id=group_id, thread_type=ThreadType.GROUP)
-        client.setTypingStatus(TypingStatus.STOPPED, thread_id=group_id, thread_type=ThreadType.GROUP)
-
-
-def start_test(param_client, param_group_id, param_user_id, tests=[]):
-    global client
-    global group_id
-    global user_id
-
-    client = param_client
-    group_id = param_group_id
-    user_id = param_user_id
-
-    tests = ['test_' + test if 'test_' != test[:5] else test for test in tests]
-
-    if len(tests) == 0:
-        suite = unittest.TestLoader().loadTestsFromTestCase(TestFbchat)
-    else:
-        suite = unittest.TestSuite(map(TestFbchat, tests))
-    print('Starting test(s)')
-    unittest.TextTestRunner(verbosity=2).run(suite)
-
-
-client = None
-
-if __name__ == '__main__':
-    # Python 3 does not use raw_input, whereas Python 2 does
-    try:
-        input = raw_input
-    except Exception as e:
-        pass
-
-    try:
-        with open(path.join(path.dirname(__file__), 'tests/my_data.json'), 'r') as f:
-            json = json.load(f)
-        email = json['email']
-        password = json['password']
-        user_id = json['user_thread_id']
-        group_id = json['group_thread_id']
-    except (IOError, IndexError) as e:
-        email = input('Email: ')
-        password = getpass()
-        group_id = input('Please enter a group thread id (To test group functionality): ')
-        user_id = input('Please enter a user thread id (To test kicking/adding functionality): ')
-
-    print('Logging in...')
-    client = CustomClient(email, password, logging_level=logging_level)
-
-    # Warning! Taking user input directly like this could be dangerous! Use only for testing purposes!
-    start_test(client, group_id, user_id, argv[1:])

tests/conftest.py

@@ -0,0 +1,101 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import unicode_literals
+
+import pytest
+import json
+
+from utils import *
+from contextlib import contextmanager
+from fbchat.models import ThreadType
+
+
+@pytest.fixture(scope="session")
+def user(client2):
+    return {"id": client2.uid, "type": ThreadType.USER}
+
+
+@pytest.fixture(scope="session")
+def group(pytestconfig):
+    return {"id": load_variable("group_id", pytestconfig.cache), "type": ThreadType.GROUP}
+
+
+@pytest.fixture(scope="session", params=["user", "group"])
+def thread(request, user, group):
+    return user if request.param == "user" else group
+
+
+@pytest.fixture(scope="session")
+def client1(pytestconfig):
+    with load_client(1, pytestconfig.cache) as c:
+        yield c
+
+
+@pytest.fixture(scope="session")
+def client2(pytestconfig):
+    with load_client(2, pytestconfig.cache) as c:
+        yield c
+
+
+@pytest.fixture  # (scope="session")
+def client(client1, thread):
+    client1.setDefaultThread(thread["id"], thread["type"])
+    yield client1
+    client1.resetDefaultThread()
+
+
+@pytest.fixture(scope="session", params=["client1", "client2"])
+def client_all(request, client1, client2):
+    return client1 if request.param == "client1" else client2
+
+
+@pytest.fixture(scope="session")
+def catch_event(client2):
+    t = ClientThread(client2)
+    t.start()
+
+    @contextmanager
+    def inner(method_name):
+        caught = CaughtValue()
+        old_method = getattr(client2, method_name)
+
+        # Will be called by the other thread
+        def catch_value(*args, **kwargs):
+            old_method(*args, **kwargs)
+            # Make sure the `set` is only called once
+            if not caught.is_set():
+                caught.set(kwargs)
+
+        setattr(client2, method_name, catch_value)
+        yield caught
+        caught.wait()
+        if not caught.is_set():
+            raise ValueError("The value could not be caught")
+        setattr(client2, method_name, old_method)
+
+    yield inner
+
+    t.should_stop.set()
+
+    try:
+        # Make the client send a messages to itself, so the blocking pull request will return
+        # This is probably not safe, since the client is making two requests simultaneously
+        client2.sendMessage("Shutdown", client2.uid)
+    finally:
+        t.join()
+
+
+@pytest.fixture  # (scope="session")
+def compare(client, thread):
+    def inner(caught_event, **kwargs):
+        d = {
+            "author_id": client.uid,
+            "thread_id": client.uid
+            if thread["type"] == ThreadType.USER
+            else thread["id"],
+            "thread_type": thread["type"],
+        }
+        d.update(kwargs)
+        return subset(caught_event.res, **d)
+
+    return inner

tests/test_base.py

@@ -0,0 +1,55 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import unicode_literals
+
+import pytest
+import py_compile
+
+from glob import glob
+from os import path, environ
+from fbchat import Client
+from fbchat.models import FBchatUserError, Message
+
+
+@pytest.mark.offline
+def test_examples():
+    # Compiles the examples, to check for syntax errors
+    for name in glob(path.join(path.dirname(__file__), "../examples", "*.py")):
+        py_compile.compile(name)
+
+
+@pytest.mark.trylast
+@pytest.mark.expensive
+def test_login(client1):
+    assert client1.isLoggedIn()
+    email = client1.email
+    password = client1.password
+
+    client1.logout()
+
+    assert not client1.isLoggedIn()
+
+    with pytest.raises(FBchatUserError):
+        client1.login("<invalid email>", "<invalid password>", max_tries=1)
+
+    client1.login(email, password)
+
+    assert client1.isLoggedIn()
+
+
+@pytest.mark.trylast
+def test_sessions(client1):
+    session = client1.getSession()
+    Client("no email needed", "no password needed", session_cookies=session)
+    client1.setSession(session)
+    assert client1.isLoggedIn()
+
+
+@pytest.mark.tryfirst
+def test_default_thread(client1, thread):
+    client1.setDefaultThread(thread["id"], thread["type"])
+    assert client1.send(Message(text="Sent to the specified thread"))
+
+    client1.resetDefaultThread()
+    with pytest.raises(ValueError):
+        client1.send(Message(text="Should not be sent"))

tests/test_fetch.py

@@ -0,0 +1,79 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import unicode_literals
+
+import pytest
+
+from os import path
+from fbchat.models import ThreadType, Message, Mention, EmojiSize, Sticker
+from utils import subset
+
+
+def test_fetch_all_users(client):
+    users = client.fetchAllUsers()
+    assert len(users) > 0
+
+
+def test_fetch_thread_list(client):
+    threads = client.fetchThreadList(limit=2)
+    assert len(threads) == 2
+
+
+@pytest.mark.parametrize(
+    "emoji, emoji_size",
+    [
+        ("😆", EmojiSize.SMALL),
+        ("😆", EmojiSize.MEDIUM),
+        ("😆", EmojiSize.LARGE),
+        # These fail because the emoji is made into a sticker
+        # This should be fixed
+        pytest.mark.xfail((None, EmojiSize.SMALL)),
+        pytest.mark.xfail((None, EmojiSize.MEDIUM)),
+        pytest.mark.xfail((None, EmojiSize.LARGE)),
+    ],
+)
+def test_fetch_message_emoji(client, emoji, emoji_size):
+    mid = client.sendEmoji(emoji, emoji_size)
+    message, = client.fetchThreadMessages(limit=1)
+
+    assert subset(
+        vars(message), uid=mid, author=client.uid, text=emoji, emoji_size=emoji_size
+    )
+
+
+def test_fetch_message_mentions(client):
+    text = "This is a test of fetchThreadMessages"
+    mentions = [Mention(client.uid, offset=10, length=4)]
+
+    mid = client.send(Message(text, mentions=mentions))
+    message, = client.fetchThreadMessages(limit=1)
+
+    assert subset(vars(message), uid=mid, author=client.uid, text=text)
+    for i, m in enumerate(mentions):
+        assert vars(message.mentions[i]) == vars(m)
+
+
+@pytest.mark.parametrize("sticker_id", ["767334476626295"])
+def test_fetch_message_sticker(client, sticker_id):
+    mid = client.send(Message(sticker=Sticker(sticker_id)))
+    message, = client.fetchThreadMessages(limit=1)
+
+    assert subset(vars(message), uid=mid, author=client.uid)
+    assert subset(vars(message.sticker), uid=sticker_id)
+
+
+def test_fetch_info(client1, group):
+    info = client1.fetchUserInfo("4")["4"]
+    assert info.name == "Mark Zuckerberg"
+
+    info = client1.fetchGroupInfo(group["id"])[group["id"]]
+    assert info.type == ThreadType.GROUP
+
+
+def test_fetch_image_url(client):
+    url = path.join(path.dirname(__file__), "image.png")
+
+    client.sendLocalImage(url)
+    message, = client.fetchThreadMessages(limit=1)
+
+    assert client.fetchImageUrl(message.attachments[0].uid)

tests/test_message_management.py

@@ -0,0 +1,12 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import unicode_literals
+
+import pytest
+
+from fbchat.models import Message, MessageReaction
+
+
+def test_set_reaction(client):
+    mid = client.send(Message(text="This message will be reacted to"))
+    client.reactToMessage(mid, MessageReaction.LOVE)

tests/test_search.py

@@ -0,0 +1,18 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import unicode_literals
+
+from fbchat.models import ThreadType
+
+
+def test_search_for(client1):
+    users = client1.searchForUsers("Mark Zuckerberg")
+    assert len(users) > 0
+
+    u = users[0]
+
+    assert u.uid == "4"
+    assert u.type == ThreadType.USER
+    assert u.photo[:4] == "http"
+    assert u.url[:4] == "http"
+    assert u.name == "Mark Zuckerberg"

tests/test_send.py

@@ -0,0 +1,110 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import unicode_literals
+
+import pytest
+
+from os import path
+from fbchat.models import Message, Mention, EmojiSize, FBchatFacebookError, Sticker
+from utils import subset
+
+
+@pytest.mark.parametrize(
+    "text",
+    [
+        "test_send",
+        "😆",
+        "\\\n\t%?&'\"",
+        "ˁҭʚ¹Ʋջوװ՞ޱɣࠚԹБɑȑңКએ֭ʗыԈٌʼőԈ×௴nચϚࠖణٔє܅Ԇޑط",
+        "a" * 20000,  # Maximum amount of characters you can send
+    ],
+)
+def test_send_text(client, catch_event, compare, text):
+    with catch_event("onMessage") as x:
+        mid = client.sendMessage(text)
+
+    assert compare(x, mid=mid, message=text)
+    assert subset(vars(x.res["message_object"]), uid=mid, author=client.uid, text=text)
+
+
+@pytest.mark.parametrize(
+    "emoji, emoji_size",
+    [
+        ("😆", EmojiSize.SMALL),
+        ("😆", EmojiSize.MEDIUM),
+        ("😆", EmojiSize.LARGE),
+        # These fail because the emoji is made into a sticker
+        # This should be fixed
+        pytest.mark.xfail((None, EmojiSize.SMALL)),
+        pytest.mark.xfail((None, EmojiSize.MEDIUM)),
+        pytest.mark.xfail((None, EmojiSize.LARGE)),
+    ],
+)
+def test_send_emoji(client, catch_event, compare, emoji, emoji_size):
+    with catch_event("onMessage") as x:
+        mid = client.sendEmoji(emoji, emoji_size)
+
+    assert compare(x, mid=mid, message=emoji)
+    assert subset(
+        vars(x.res["message_object"]),
+        uid=mid,
+        author=client.uid,
+        text=emoji,
+        emoji_size=emoji_size,
+    )
+
+
+@pytest.mark.xfail(raises=FBchatFacebookError)
+@pytest.mark.parametrize("message", [Message("a" * 20001)])
+def test_send_invalid(client, message):
+    client.send(message)
+
+
+def test_send_mentions(client, client2, thread, catch_event, compare):
+    text = "Hi there @me, @other and @thread"
+    mentions = [
+        dict(thread_id=client.uid, offset=9, length=3),
+        dict(thread_id=client2.uid, offset=14, length=6),
+        dict(thread_id=thread["id"], offset=26, length=7),
+    ]
+    with catch_event("onMessage") as x:
+        mid = client.send(Message(text, mentions=[Mention(**d) for d in mentions]))
+
+    assert compare(x, mid=mid, message=text)
+    assert subset(vars(x.res["message_object"]), uid=mid, author=client.uid, text=text)
+    # The mentions are not ordered by offset
+    for m in x.res["message_object"].mentions:
+        assert vars(m) in mentions
+
+
+@pytest.mark.parametrize(
+    "sticker_id",
+    ["767334476626295", pytest.mark.xfail("0", raises=FBchatFacebookError)],
+)
+def test_send_sticker(client, catch_event, compare, sticker_id):
+    with catch_event("onMessage") as x:
+        mid = client.send(Message(sticker=Sticker(sticker_id)))
+
+    assert compare(x, mid=mid)
+    assert subset(vars(x.res["message_object"]), uid=mid, author=client.uid)
+    assert subset(vars(x.res["message_object"].sticker), uid=sticker_id)
+
+
+@pytest.mark.parametrize(
+    "method_name, url",
+    [
+        (
+            "sendRemoteImage",
+            "https://github.com/carpedm20/fbchat/raw/master/tests/image.png",
+        ),
+        ("sendLocalImage", path.join(path.dirname(__file__), "image.png")),
+    ],
+)
+def test_send_images(client, catch_event, compare, method_name, url):
+    text = "An image sent with {}".format(method_name)
+    with catch_event("onMessage") as x:
+        mid = getattr(client, method_name)(url, Message(text))
+
+    assert compare(x, mid=mid, message=text)
+    assert subset(vars(x.res["message_object"]), uid=mid, author=client.uid, text=text)
+    assert x.res["message_object"].attachments[0]

tests/test_tests.py

@@ -0,0 +1,12 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import unicode_literals
+
+import pytest
+
+
+def test_catch_event(client2, catch_event):
+    mid = "test"
+    with catch_event("onMessage") as x:
+        client2.onMessage(mid=mid)
+    assert x.res['mid'] == mid

tests/test_thread_interraction.py

@@ -0,0 +1,100 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import unicode_literals
+
+import pytest
+
+from fbchat.models import (
+    Message,
+    ThreadType,
+    FBchatFacebookError,
+    TypingStatus,
+    ThreadColor,
+)
+from utils import random_hex, subset
+from os import environ
+
+
+def test_remove_from_and_add_to_group(client1, client2, group, catch_event):
+    # Test both methods, while ensuring that the user gets added to the group
+    try:
+        with catch_event("onPersonRemoved") as x:
+            client1.removeUserFromGroup(client2.uid, group["id"])
+        assert subset(
+            x.res, removed_id=client2.uid, author_id=client1.uid, thread_id=group["id"]
+        )
+    finally:
+        with catch_event("onPeopleAdded") as x:
+            client1.addUsersToGroup(client2.uid, group["id"])
+        assert subset(
+            x.res, added_ids=[client2.uid], author_id=client1.uid, thread_id=group["id"]
+        )
+
+
+@pytest.mark.xfail(
+    raises=FBchatFacebookError, reason="Apparently changeThreadTitle is broken"
+)
+def test_change_title(client1, catch_event, group):
+    title = random_hex()
+    with catch_event("onTitleChange") as x:
+        mid = client1.changeThreadTitle(title, group["id"])
+    assert subset(
+        x.res,
+        mid=mid,
+        author_id=client1.uid,
+        new_title=title,
+        thread_id=group["id"],
+        thread_type=ThreadType.GROUP,
+    )
+
+
+def test_change_nickname(client, client_all, catch_event, compare):
+    nickname = random_hex()
+    with catch_event("onNicknameChange") as x:
+        client.changeNickname(nickname, client_all.uid)
+    assert compare(x, changed_for=client_all.uid, new_nickname=nickname)
+
+
+@pytest.mark.parametrize("emoji", ["😀", "😂", "😕", "😍"])
+def test_change_emoji(client, catch_event, compare, emoji):
+    with catch_event("onEmojiChange") as x:
+        client.changeThreadEmoji(emoji)
+    assert compare(x, new_emoji=emoji)
+
+
+@pytest.mark.xfail(raises=FBchatFacebookError)
+@pytest.mark.parametrize("emoji", ["🙃", "not an emoji"])
+def test_change_emoji_invalid(client, emoji):
+    client.changeThreadEmoji(emoji)
+
+
+@pytest.mark.parametrize(
+    "color",
+    [
+        x
+        if x in [ThreadColor.MESSENGER_BLUE, ThreadColor.PUMPKIN]
+        else pytest.mark.expensive(x)
+        for x in ThreadColor
+    ],
+)
+def test_change_color(client, catch_event, compare, color):
+    with catch_event("onColorChange") as x:
+        client.changeThreadColor(color)
+    assert compare(x, new_color=color)
+
+
+@pytest.mark.xfail(
+    raises=FBchatFacebookError, strict=False, reason="Should fail, but doesn't"
+)
+def test_change_color_invalid(client):
+    class InvalidColor:
+        value = "#0077ff"
+
+    client.changeThreadColor(InvalidColor())
+
+
+@pytest.mark.parametrize("status", TypingStatus)
+def test_typing_status(client, catch_event, compare, status):
+    with catch_event("onTyping") as x:
+        client.setTypingStatus(status)
+    assert compare(x, status=status)

tests/utils.py

@@ -0,0 +1,83 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import unicode_literals
+
+import threading
+import logging
+import six
+
+from os import environ
+from random import randrange
+from contextlib import contextmanager
+from six import viewitems
+from fbchat import Client
+from fbchat.models import ThreadType
+
+log = logging.getLogger("fbchat.tests").addHandler(logging.NullHandler())
+
+
+class ClientThread(threading.Thread):
+    def __init__(self, client, *args, **kwargs):
+        self.client = client
+        self.should_stop = threading.Event()
+        super(ClientThread, self).__init__(*args, **kwargs)
+
+    def start(self):
+        self.client.startListening()
+        self.client.doOneListen()  # QPrimer, Facebook now knows we're about to start pulling
+        super(ClientThread, self).start()
+
+    def run(self):
+        while not self.should_stop.is_set() and self.client.doOneListen():
+            pass
+
+        self.client.stopListening()
+
+
+if six.PY2:
+    event_class = threading._Event
+else:
+    event_class = threading.Event
+
+
+class CaughtValue(event_class):
+    def set(self, res):
+        self.res = res
+        super(CaughtValue, self).set()
+
+    def wait(self, timeout=3):
+        super(CaughtValue, self).wait(timeout=timeout)
+
+
+def random_hex(length=20):
+    return "{:X}".format(randrange(16 ** length))
+
+
+def subset(a, **b):
+    print(a)
+    print(b)
+    return viewitems(b) <= viewitems(a)
+
+
+def load_variable(name, cache):
+    var = environ.get(name, None)
+    if var is not None:
+        if cache.get(name, None) != var:
+            cache.set(name, var)
+        return var
+
+    var = cache.get(name, None)
+    if var is None:
+        raise ValueError("Variable {!r} neither in environment nor cache".format(name))
+    return var
+
+
+@contextmanager
+def load_client(n, cache):
+    client = Client(
+        load_variable("client{}_email".format(n), cache),
+        load_variable("client{}_password".format(n), cache),
+        session_cookies=cache.get("client{}_session".format(n), None),
+    )
+    yield client
+    cache.set("client{}_session".format(n), client.getSession())