-
Notifications
You must be signed in to change notification settings - Fork 429
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Send users notification when api flags have been blocked #4338
Merged
matthewelwell
merged 11 commits into
main
from
feat/send_users_notification_when_api_flags_have_been_blocked
Jul 25, 2024
Merged
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
a766788
Create notification templates
zachaysan 64cbf69
Create and call send_api_flags_blocked_notification
zachaysan 619ecc1
Add email tests to existing test structure
zachaysan e96612c
Move non-task functions into helpers module
zachaysan 1fccd8b
Update failing tests from task helpers move
zachaysan ebf3cf6
Fix conflicts and merge branch 'main' into feat/send_users_notificati…
zachaysan 9cfb448
Switch tests to use render_to_string for templates
zachaysan 5c8f60b
Update flags blocked email notifications
zachaysan 84efedb
Test when usage threshold is below minimum triggerable threshold
zachaysan 07ca079
Update function privacy to current usage
zachaysan c8cd92a
Merge branch 'main' into feat/send_users_notification_when_api_flags_…
zachaysan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,137 @@ | ||
import logging | ||
from datetime import timedelta | ||
|
||
from app_analytics.influxdb_wrapper import get_current_api_usage | ||
from dateutil.relativedelta import relativedelta | ||
from django.conf import settings | ||
from django.core.mail import send_mail | ||
from django.template.loader import render_to_string | ||
from django.utils import timezone | ||
|
||
from organisations.models import ( | ||
Organisation, | ||
OrganisationAPIUsageNotification, | ||
OrganisationRole, | ||
) | ||
from users.models import FFAdminUser | ||
|
||
from .constants import API_USAGE_ALERT_THRESHOLDS | ||
|
||
logger = logging.getLogger(__name__) | ||
|
||
|
||
def send_api_flags_blocked_notification(organisation: Organisation) -> None: | ||
recipient_list = FFAdminUser.objects.filter( | ||
userorganisation__organisation=organisation, | ||
) | ||
|
||
context = {"organisation": organisation} | ||
message = "organisations/api_flags_blocked_notification.txt" | ||
html_message = "organisations/api_flags_blocked_notification.html" | ||
|
||
send_mail( | ||
subject="Flagsmith API use has been blocked due to overuse", | ||
message=render_to_string(message, context), | ||
from_email=settings.DEFAULT_FROM_EMAIL, | ||
recipient_list=list(recipient_list.values_list("email", flat=True)), | ||
html_message=render_to_string(html_message, context), | ||
fail_silently=True, | ||
) | ||
|
||
|
||
def send_api_usage_notification( | ||
organisation: Organisation, matched_threshold: int | ||
) -> None: | ||
""" | ||
Send notification to users that the API has breached a threshold. | ||
|
||
Only admins are included if the matched threshold is under | ||
100% of the API usage limits. | ||
""" | ||
|
||
recipient_list = FFAdminUser.objects.filter( | ||
userorganisation__organisation=organisation, | ||
) | ||
|
||
if matched_threshold < 100: | ||
message = "organisations/api_usage_notification.txt" | ||
html_message = "organisations/api_usage_notification.html" | ||
|
||
# Since threshold < 100 only include admins. | ||
recipient_list = recipient_list.filter( | ||
userorganisation__role=OrganisationRole.ADMIN, | ||
) | ||
else: | ||
message = "organisations/api_usage_notification_limit.txt" | ||
html_message = "organisations/api_usage_notification_limit.html" | ||
|
||
context = { | ||
"organisation": organisation, | ||
"matched_threshold": matched_threshold, | ||
} | ||
|
||
send_mail( | ||
subject=f"Flagsmith API use has reached {matched_threshold}%", | ||
message=render_to_string(message, context), | ||
from_email=settings.DEFAULT_FROM_EMAIL, | ||
recipient_list=list(recipient_list.values_list("email", flat=True)), | ||
html_message=render_to_string(html_message, context), | ||
fail_silently=True, | ||
) | ||
|
||
OrganisationAPIUsageNotification.objects.create( | ||
organisation=organisation, | ||
percent_usage=matched_threshold, | ||
notified_at=timezone.now(), | ||
) | ||
|
||
|
||
def _handle_api_usage_notifications(organisation: Organisation) -> None: | ||
now = timezone.now() | ||
|
||
if organisation.subscription.is_free_plan: | ||
allowed_api_calls = organisation.subscription.max_api_calls | ||
# Default to a rolling month for free accounts | ||
days = 30 | ||
period_starts_at = now - timedelta(days) | ||
elif not organisation.has_subscription_information_cache(): | ||
# Since the calling code is a list of many organisations | ||
# log the error and return without raising an exception. | ||
logger.error( | ||
f"Paid organisation {organisation.id} is missing subscription information cache" | ||
) | ||
return | ||
else: | ||
subscription_cache = organisation.subscription_information_cache | ||
billing_starts_at = subscription_cache.current_billing_term_starts_at | ||
|
||
# Truncate to the closest active month to get start of current period. | ||
month_delta = relativedelta(now, billing_starts_at).months | ||
period_starts_at = relativedelta(months=month_delta) + billing_starts_at | ||
|
||
days = relativedelta(now, period_starts_at).days | ||
allowed_api_calls = subscription_cache.allowed_30d_api_calls | ||
|
||
api_usage = get_current_api_usage(organisation.id, f"-{days}d") | ||
|
||
api_usage_percent = int(100 * api_usage / allowed_api_calls) | ||
|
||
matched_threshold = None | ||
for threshold in API_USAGE_ALERT_THRESHOLDS: | ||
if threshold > api_usage_percent: | ||
break | ||
|
||
matched_threshold = threshold | ||
|
||
# Didn't match even the lowest threshold, so no notification. | ||
if matched_threshold is None: | ||
return | ||
|
||
if OrganisationAPIUsageNotification.objects.filter( | ||
notified_at__gt=period_starts_at, | ||
percent_usage__gte=matched_threshold, | ||
).exists(): | ||
# Already sent the max notification level so don't resend. | ||
return | ||
|
||
send_api_usage_notification(organisation, matched_threshold) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,11 +3,8 @@ | |
from datetime import timedelta | ||
|
||
from app_analytics.influxdb_wrapper import get_current_api_usage | ||
from dateutil.relativedelta import relativedelta | ||
from django.conf import settings | ||
from django.core.mail import send_mail | ||
from django.db.models import F, Max | ||
from django.template.loader import render_to_string | ||
from django.utils import timezone | ||
from task_processor.decorators import ( | ||
register_recurring_task, | ||
|
@@ -25,7 +22,6 @@ | |
Organisation, | ||
OrganisationAPIBilling, | ||
OrganisationAPIUsageNotification, | ||
OrganisationRole, | ||
Subscription, | ||
) | ||
from organisations.subscriptions.constants import FREE_PLAN_ID | ||
|
@@ -37,7 +33,6 @@ | |
from .constants import ( | ||
ALERT_EMAIL_MESSAGE, | ||
ALERT_EMAIL_SUBJECT, | ||
API_USAGE_ALERT_THRESHOLDS, | ||
API_USAGE_GRACE_PERIOD, | ||
) | ||
from .subscriptions.constants import ( | ||
|
@@ -47,6 +42,10 @@ | |
STARTUP_V2, | ||
SubscriptionCacheEntity, | ||
) | ||
from .task_helpers import ( | ||
_handle_api_usage_notifications, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This shouldn't still be a private function since we're calling it from another module now. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok I've renamed it |
||
send_api_flags_blocked_notification, | ||
) | ||
|
||
logger = logging.getLogger(__name__) | ||
|
||
|
@@ -105,104 +104,7 @@ def finish_subscription_cancellation() -> None: | |
subscription.save_as_free_subscription() | ||
|
||
|
||
def send_api_usage_notification( | ||
organisation: Organisation, matched_threshold: int | ||
) -> None: | ||
""" | ||
Send notification to users that the API has breached a threshold. | ||
|
||
Only admins are included if the matched threshold is under | ||
100% of the API usage limits. | ||
""" | ||
|
||
recipient_list = FFAdminUser.objects.filter( | ||
userorganisation__organisation=organisation, | ||
) | ||
|
||
if matched_threshold < 100: | ||
message = "organisations/api_usage_notification.txt" | ||
html_message = "organisations/api_usage_notification.html" | ||
|
||
# Since threshold < 100 only include admins. | ||
recipient_list = recipient_list.filter( | ||
userorganisation__role=OrganisationRole.ADMIN, | ||
) | ||
else: | ||
message = "organisations/api_usage_notification_limit.txt" | ||
html_message = "organisations/api_usage_notification_limit.html" | ||
|
||
context = { | ||
"organisation": organisation, | ||
"matched_threshold": matched_threshold, | ||
} | ||
|
||
send_mail( | ||
subject=f"Flagsmith API use has reached {matched_threshold}%", | ||
message=render_to_string(message, context), | ||
from_email=settings.DEFAULT_FROM_EMAIL, | ||
recipient_list=list(recipient_list.values_list("email", flat=True)), | ||
html_message=render_to_string(html_message, context), | ||
fail_silently=True, | ||
) | ||
|
||
OrganisationAPIUsageNotification.objects.create( | ||
organisation=organisation, | ||
percent_usage=matched_threshold, | ||
notified_at=timezone.now(), | ||
) | ||
|
||
|
||
def _handle_api_usage_notifications(organisation: Organisation) -> None: | ||
now = timezone.now() | ||
|
||
if organisation.subscription.is_free_plan: | ||
allowed_api_calls = organisation.subscription.max_api_calls | ||
# Default to a rolling month for free accounts | ||
days = 30 | ||
period_starts_at = now - timedelta(days) | ||
elif not organisation.has_subscription_information_cache(): | ||
# Since the calling code is a list of many organisations | ||
# log the error and return without raising an exception. | ||
logger.error( | ||
f"Paid organisation {organisation.id} is missing subscription information cache" | ||
) | ||
return | ||
else: | ||
subscription_cache = organisation.subscription_information_cache | ||
billing_starts_at = subscription_cache.current_billing_term_starts_at | ||
|
||
# Truncate to the closest active month to get start of current period. | ||
month_delta = relativedelta(now, billing_starts_at).months | ||
period_starts_at = relativedelta(months=month_delta) + billing_starts_at | ||
|
||
days = relativedelta(now, period_starts_at).days | ||
allowed_api_calls = subscription_cache.allowed_30d_api_calls | ||
|
||
api_usage = get_current_api_usage(organisation.id, f"-{days}d") | ||
|
||
api_usage_percent = int(100 * api_usage / allowed_api_calls) | ||
|
||
matched_threshold = None | ||
for threshold in API_USAGE_ALERT_THRESHOLDS: | ||
if threshold > api_usage_percent: | ||
break | ||
|
||
matched_threshold = threshold | ||
|
||
# Didn't match even the lowest threshold, so no notification. | ||
if matched_threshold is None: | ||
return | ||
|
||
if OrganisationAPIUsageNotification.objects.filter( | ||
notified_at__gt=period_starts_at, | ||
percent_usage__gte=matched_threshold, | ||
).exists(): | ||
# Already sent the max notification level so don't resend. | ||
return | ||
|
||
send_api_usage_notification(organisation, matched_threshold) | ||
|
||
|
||
# Task enqueued in register_recurring_tasks below. | ||
def handle_api_usage_notifications() -> None: | ||
flagsmith_client = get_client("local", local_eval=True) | ||
|
||
|
@@ -228,6 +130,7 @@ def handle_api_usage_notifications() -> None: | |
) | ||
|
||
|
||
# Task enqueued in register_recurring_tasks below. | ||
def charge_for_api_call_count_overages(): | ||
now = timezone.now() | ||
|
||
|
@@ -332,6 +235,7 @@ def charge_for_api_call_count_overages(): | |
) | ||
|
||
|
||
# Task enqueued in register_recurring_tasks below. | ||
def restrict_use_due_to_api_limit_grace_period_over() -> None: | ||
""" | ||
Restrict API use once a grace period has ended. | ||
|
@@ -389,6 +293,9 @@ def restrict_use_due_to_api_limit_grace_period_over() -> None: | |
organisation.stop_serving_flags = stop_serving | ||
organisation.block_access_to_admin = block_access | ||
|
||
if stop_serving: | ||
send_api_flags_blocked_notification(organisation) | ||
|
||
# Save models individually to allow lifecycle hooks to fire. | ||
organisation.save() | ||
|
||
|
@@ -397,6 +304,7 @@ def restrict_use_due_to_api_limit_grace_period_over() -> None: | |
APILimitAccessBlock.objects.bulk_create(api_limit_access_blocks) | ||
|
||
|
||
# Task enqueued in register_recurring_tasks below. | ||
def unrestrict_after_api_limit_grace_period_is_stale() -> None: | ||
""" | ||
This task handles accounts that have breached the API limit | ||
|
30 changes: 30 additions & 0 deletions
30
api/organisations/templates/organisations/api_flags_blocked_notification.html
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
<table> | ||
|
||
<tr> | ||
|
||
<td>Hi there,</td> | ||
|
||
</tr> | ||
|
||
<tr> | ||
|
||
<td> | ||
This is a system generated notification related to your Flagsmith API Usage. As per previous warnings, we have had to block your company {{ organisation.name }} after the 7 day grace period. Flags are not currently being served for your organization, and will continue to be blocked until your billing period resets or you upgrade your account. You can upgrade your account at <a href="app.flagsmith.com">app.flagsmith.com</a>. | ||
</td> | ||
|
||
|
||
</tr> | ||
|
||
<tr> | ||
|
||
<td>Thank you!</td> | ||
|
||
</tr> | ||
|
||
<tr> | ||
|
||
<td>The Flagsmith Team</td> | ||
|
||
</tr> | ||
|
||
</table> |
7 changes: 7 additions & 0 deletions
7
api/organisations/templates/organisations/api_flags_blocked_notification.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
Hi there, | ||
|
||
This is a system generated notification related to your Flagsmith API Usage. As per previous warnings, we have had to block your company {{ organisation.name }} after the 7 day grace period. Flags are not currently being served for your organization, and will continue to be blocked until your billing period resets or you upgrade your account. You can upgrade your account at app.flagsmith.com. | ||
|
||
Thank you! | ||
|
||
The Flagsmith Team |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This probably can be a private function now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I've renamed it.