* ORDER BY referrer_source for spikes job
This is more consistent with the rest of the queries
* Refactor top_sources -> top_sources_for_spike
* Remove more dead code
* Remove unused arguments
* Remove unused select arguments
* Add a test to top_sources_for_spike
* Add Ecto.Network dependency
* Migration: Add ip block list table
* If Cachex errors out, mark the cache as not ready
* Add IPRule schema
* Seed IPRules
* Add Shields context module
* Implement IPRuleCache
* Start IPRuleCache
* Drop blocklisted IPs on ingestion
* Cosmetic rename
* Add settings sidebar item
* Consider IPRuleCache readiness on health checks
* Fix typo
* Implement IP blocklist live view
* Update moduledocs
* Extend contextual module tests
* Convert IPRules LiveView into LiveComponent
* Keep live flashes on the tabs view
* Update changelog
* Format
* Credo
* Remove garbage
* Update drop reason typespecs
* Update typespecs for cache keys
* Keep track of who added a rule and when
* Test if adding via LV prefills the updated_by tooltip
* Update ecto_network dependency
* s/updated_by/added_by
* s/drop_blocklist_ip/drop_shield_rule_ip
* Add docs link
* s/Updated/Added
* WIP: PropFilterRow
* Get multi-behavior working
* Render multiple prop filters in one
* Modal reads from query string correctly
* Backend support for multiple custom property filters
* Add backend tests for multiple custom property filters
* Disable already selected options in property keys
We can't allow choosing the same property multiple times without changing the request
params, which we decided against
* Allow choosing any property under Behaviors > Custom props even if custom prop filter applied
This was a limitation (I believe) introduced by using ARRAY JOINs to query custom properties
* CHANGELOG.md
* Solve credo warning about too deep nesting
* Update assets/js/dashboard/stats/modals/prop-filter-modal.js
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
* Refactor internal function for clarity
* Add another step -> Add another
* Solve 500 error
* Separate boxes per property filter
* Retain other filters in props table
* removeFilter behavior for props
* matches_member support for custom props
* filter_suggestions for prop keys should account for prop filter
* find over filter
* refactor appliedFilters
* FILTER_TYPES => FILTER_OPERATIONS
* Make add another link not wrap the whole page
* Unique keys
---------
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
* Reorganize how subscriptions/trials are evaluated
* Bugfix: expired trial+no subscriptions should not have access to extra features
* Make self-hosted users always on trial
* Seed secondary user with password
* Format
* Fix docs
* Fix small_test run
* Run the test only on full_build
* More tweaks to small builds
* Allow [Goals] for expired trials with no subscription
* Simplify caches config
The intervals were proven to never change really,
and previous shape made them not very discoverable either.
We'll look into a more declarative setup.
* Aim for more declarative supervision tree setup
* Generalize `Plausible.Cache`
This makes the `Site.Cache` module implement the specifics,
leaving the common bits reusable for upcoming cache processes.
* Generalize `Plausible.Cache` warmers
* Fix typos
* Revert "Remove 0-sized queue"
This reverts commit 6d229b3f70.
* Revert "Temporarily remove notification cron worker (#3732)"
This reverts commit 447e3929cb.
* Revert "Temporarily disable accept_traffic_until actions (#3730)"
This reverts commit 58afe3376f.
* Send accept_traffic_until notifications at 8 UTC rather than 5
* Add option for dry-run
* Expose dry_run in a dedicated function
* add tests for filtering by goal in timeseries and aggregate
* refactor filter parsing
* stop returning custom props in event:goal breakdown
* test breaking down wildcard pageview goals
* extract filter utils
* parse more goal filter options
* add passing tests for new filter types
* do not allow querying session metrics with a goal filter
* remove unused page_match property
* test that non-configured goals are not returned in breakdown
* enforce filtered goals configured
* update changelog
* Allow simple filtering by revenue goals
This does not mean that revenue metrics are supported. If a revenue goal
is filtered by, we treat it like a simple custom event goal in the API.
* use List.wrap
* Make subsequent query boundaries idempotent
This should prevent time-sensitive race conditions
on subsequent `Plausible.Stats.*` calls for the same query.
Not sure if there are more places where now() is
dynamic relative to query life cycle, but it's a start.
* Format
* Update lib/plausible/stats/query.ex
Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>
* Match on `Query.now` in a test
---------
Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>
* Move imported tables schemas to separate modules outside Google ns
* Move buffer for imports to Imported ns
* fix schema newlines
* Extract UA import processing and persistence
* Decouple analytics worker implementation from UA
* Rename env variable for import buffer size
* Preserve old import queue until release
* Use array indexing instead of left joins
This has multiple benefits:
- Allows us to more easily add support for multiple custom property
filters
- Performance: Increased performance of some queries by ~20%
Work left to do:
- Check tests coverage
- Create macros for repeated fragments
- Quote and refer to right tables in fragments/macros
* Use macros to clean up code
* Use existing atoms to avoid memory leaks
* Enable breakdown property tracing for event:goal
This is currently overwritten by event:name. The queries for event:name
will still be tracked if queried specifically.
* Add filters tracing to Query.trace/1
* make sure breakdown metrics are sorted and joined
* check opts skip tracing directly without passing it
* Allow `matches` operator to work in BE for custom props
Note: No FE support yet, needs further testing
* feat: allow choosing `contains` for property filters in the UI
* no autocomplete on prop values if `contains` for consistency
* CHANGELOG.md
* Fix: Handle (none) property in property breakdowns when using matching
When matching we should always exclude (none)
* Remove allowance_required field from grace_period
Since we are now preventing customers from subscribing to a plan that
does not accommodate their pageview usage, there is no need for an extra
check on removing the grace period after a successful upgrade.
This extra check is the reason why the automatic unlocks have recently
failed in several cases.
* refactor outgrown subscription notices
* make a test actually test the described functionality
* Apply greater pageview allowance margin only for trial upgrades
...in order to prevent cancelled or paused subscriptions from subscribing
to plans that would still leave their account locked.
* Mark the entire ChoosePlanTest module full build only
* remove account locking guide
This is irrelevant for self-hosters, and the internal knowledge base is
a better place for this document. Moved it there.
* refactor Keyword get clause
* add a pattern matching assertion in code
* Fix a bug
This commit fixes a bug where the timeseries weekly interval query for a
month returned 0 visitors for the first week.
This happened because the start of the month defining the first week was
missing the UTC -> site.timezone conversion.
* Update test/plausible_web/controllers/api/stats_controller/main_graph_test.exs
Remove dot from test description
Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
---------
Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
* Fall back to UTC when timezone gap found
* bugfix 1
* bugfix 2
* Apply safe tz conversion to all cases where it's done on a ts other than now
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Display premium features tabs on the dashboard
This commit makes the funnels and props tab on the dashboard visible
even when the site owner's plan does not have access to that feature.
This is to raise awareness of those features, and if the site owner does
not want to see that tab, they can still click "Hide this report" to
hide it.
Previously, when the plan did not support a feature, the feature module
`enabled/1` function returned `false`, regardless of the
`sites.feature_enabled` toggle. This commit creates a new function
called `opted_out/1` to differentiate access from explicitly opting out
a feature.
* Remove unused data-conversions-enabled attribute
* DRY Plausible.Billing.Feature.check_availability/1 function
* Allow opting out features the user doesn't have access to
* add upgrade CTA to FeatureSetupNotice
* fix JS linting errors
* simplify notice.js
* fix behaviour when deleting funnels saved to localstorage
In case some other funnel exists, we will use that as the default
selected one. If not, a feature setup notice will be displayed again.
---------
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* remove unused code
* add e.name == "pageview" condition to pageview goals
This fixes the weird behavior where filtering by a pageview goal would
also return custom events and vice versa.
* update changelog
* Update communication
* Remove an unreachable function (mistyped)
* [migration] Make accept_traffic_until a date
* Fix typo
* Set `accept_traffic_until` when creating a site
* Update sites `accept_traffic_until` on subscription change
* Add a note to yearly cancellation notification
* Rephrase annual e-mail for clarity
* Pass the small build test
* Add email notifications
* Fixup
* Implement `accept_traffic_until` notification worker
* Fixup - no need to test this for small build
* Update moduledoc
* Move moduletag
* s/sent_at/sent_on
* Use WHERE NOT EXISTS instead of LEFT JOIN
* Use upsert when tracking notifications sent
* Store sent marker before actually sending notification
* Prefer to keep `accept_traffic_until` on the user record
This gives us a single source of truth, addresses cases like
ownership transparently, simplifies the code and enables CRM toggles.
The only downside is that there's another join performed in the
Sites.Cache full refresh - in this case, small refreshes are
skipped - but this is fine, since the traffic will be let in
anyway.
* Expose `accepted_traffic_until` in the CRM
* Update lib/plausible/auth/user.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Preload owner in CRM
* Use the offset parameter in trial over e-mail contents
* Format
* Harden cache test
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* use more convenient testing functions
* do not display + sign with 0% change in emails
* Rename module/file/function names
before, `weekly_report` was also used for monthly reports and that was a
bit confusing to read in code.
* Refactor send_email_report.ex
This commit improves readability by refactoring the code into smaller
functions and reducing the number of arguments given to functions.
But more importantly, it stops making duplicate stats queries for every
email recipient by moving the queries out of the for loop.
* Refactor: move querying logic out of the worker module
and merge all stats information under a single `stats` assign.
* Move limit enforcement to accepting site ownerhsip transfer
* enforce pageview limit on ownership transfer accept
* Refactor plan limit check logic
* Extract `ensure_can_take_ownership` to `Invitations` context and refactor
* Improve styling of exceeded limits notice in invitation dialog and disable button
* styling improvements to notice
* make transfer_ownership return transfer to self error
* do not allow transferring to user without active subscription WIP
* Add missing typespec and improve existing ones
* Fix formatting
* Explicitly label direct match on function argument for clarity
* Slightly refactor `CreateInvitation.bulk_transfer_ownership_direct`
* Exclude quota enforcement tests from small build test suite
* Remove unused return type from `invite_error()` union type
* Do not block plan upgrade when there's pending ownership transfer
* Don't block and only warn about missing features on transfer
* Remove `x-init` attribute used for debugging
* Add tests for `Quota.monthly_pageview_usage/2`
* Test and improve site admin ownership transfer actions
* Extend tests for `AcceptInvitation.transfer_ownership`
* Test transfer ownership controller level accept action error cases
* Test choosing plan by user without sites but with a pending ownership transfer
* Test invitation x-data in sites LV
* Remove sitelocker trigger in invitation acceptance code and simplify logic
* Add Quota test for `user.allow_next_upgrade_override` being set
* ignore pageview limit only when subscribing to plan
* Use sandbox Paddle instance for staging
* Use sandbox paddle key for staging and dev
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* move format_price to Plausible.Billing
* move PlausibleWeb.Components.Billing file to subfolder
* extract new Notice module
* rename test file and module name
* move growth_grandfathered notice to notice.ex
* extract a PlanBenefits module
* extract PlanBox component
* extract PageviewSlider component
* fix plan benefits text color
* Remove business tier feature flag
This commit removes all code branches related to the business tier
feature flag, as we're not flipping this flag off anymore. It also
removes unused routes, e.g. /billing/change-plan and /billing/upgrade
* remove unused billing templates
* refactor with clause to case instead
* assert on the url in email tests
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* remove unused functionality
We can now safely delete all the logic around users who are on trial and
signed up *before* the business tiers release, since that's no longer
possible
* add monthly_pageview_limit fn clause that takes a user
* make Quota.site_limit return enterprise site limit
...not `:unlimited`, as we still need to display it
in the account settings.
* make `team_member_limit/1` return :unlimited on small_build
* improve team_member_usage/1 function doc
* stop displaying unlimited symbol in usage section
These unlimited limits include:
* `monthly_pageview_limit` for trials
* `team_member_limit` for old enterprise plans
* `site limit` for old accounts (before 2021-05-05)
* format
* small refactor case clause + move mod vars
* review suggestions
* Add data retention field to plans
* Display data retention as benefit when choosing plan
* Split fields in two module attributes
* Remove extra whitespace
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Improve subscription status checking code
This commit improves the subscription status checking in code, and
creates convinience functions to work with it, including nil-checking
and in?/2 function.
* Change in?/2 to macro