Bridging the SAML and OAuth 2.0 frameworks is a well understood problem. The following stack of IETF specs provides a standard solution:
If you look at the core OAuth 2.0 spec (RFC 6749) and its token endpoint definition - this is basically an OAuth server endpoint which returns an access token in exchange for a "grant" -- an open-ended concept of something deemed appropriate to grant the client app the issue of an access token. In the typical OAuth scenario this is an authorisation code signifying that the user has been previously authenticated and given their consent. But the grant could also be something else.
There is a further IETF spec called draft-ietf-oauth-assertions-16 that builds on the core RFC 6749 standard which says that the grant can also be an assertion (a signed proof of something) and defines the necessary token request parameters for that.
Finally, there is draft-ietf-oauth-saml2-bearer-20, which specifies how this assertion can be a SAML 2.0 Bearer Assertion.
This standard mechanism for converting a SAML assertion into an OAuth 2.0 access token is essentially all that is needed to bridge the two frameworks.
To ensure removal of users is properly reflected by the authorisation systems there are two approaches, which can be combined:
Make the OAuth 2.0 access tokens short lived. This will force the client to repeat the authorisation process when the token expires, and if the user no longer exists authentication will fail and no grant (SAML assertion) will be issued.
Provide an API for revoking issued OAuth 2.0 access tokens, see RFC 7009 for details.