Commit Graph

783 Commits (con-cache-patch-test)

Author SHA1 Message Date
Karl-Aksel Puulmann 5cdca6f408
Remove code that updates session browser/geo/OS attributes (#3796)
This will makes it impossible to store session attributes on events.

Looking at production data also revealed no cases where these updates
are effective.
3 years ago
hq1 eceac8afd5
Allow inviting users who are members already (#3797)
* Allow e-mail exclusion in team members quota

* Exclude invitee from quota on invitation create

* Enable invitation submission but report errors on quota violation

* Use a single interface for team members quota

* Check the `Keyword.validate/2` result

* Update test/plausible_web/controllers/site/membership_controller_test.exs

Co-authored-by: Uku Taht <Uku.taht@gmail.com>

---------

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
3 years ago
RobertJoonas 8e8790dd30
Revert api conversion rate (#3789)
* Revert "Unify percentage change for CR and bounce_rate (#3781)"

This reverts commit a6b1a6ebc7.

* Revert "Bring Stats API up to speed: Add `conversion_rate` to Aggregate and Breakdown (#3739)"

This reverts commit 672d682e95.
3 years ago
RobertJoonas a6b1a6ebc7
Unify percentage change for CR and bounce_rate (#3781)
* Fix conversion rate change calculation

The change in conversion rate should be calculated similar to bounce rate.
For example, an increase of 25% -> 50% should not be a 100% change, but
a 25% change instead.

* Use the same comparison function in Stats API and dashboard API

This commit fixes a bug where the percentage change reported by the Stats
API is different from the one returned by the internal dashboard API.

* changelog update
3 years ago
hq1 926de4dd10
Experimental session count (#3786)
* WIP

* Allow `experimetnal_session_count` request serialization

* Extend `Plausible.Stats.Query` with `experimental_session_count` flag

* Add `FunWithFlags` actor implementation for `Site`

* Change the way sessions are retrieved

* Remove redundant test

* Format

* Update the test

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
3 years ago
RobertJoonas 672d682e95
Bring Stats API up to speed: Add `conversion_rate` to Aggregate and Breakdown (#3739)
* disable event metric with include_imported in every case

* add missing test for metric validation

* refactor metric validation functions

* implement conversion_rate metric validation

* move calculate_cr function into Stats.Util

* Refactor: Move aggregate CR logic into Stats.aggregate

* define atoms to exist

* Ensure that CR does not depend on visitors being queried

If 'visitors' are already queried, we'll use that value. Otherwise we'll
need to make another query to fetch it.

* confirm Stats API aggregate supports CR (tests only)

* small refactor

This is the only 'event_property' left after pattern matching on all
others in the function clauses defined above.

* Make it possible to optionally query conversion_rate

...in breakdown queries (excluding goal and custom prop breakdown)

* A little refactor asking for revenue metrics

1. The `@revenue_metrics` module attribute is an empty list on full build
   anyway
2. We don't need to query for revenue metrics if there are no revenue goals
   returned in the given query (even if revenue goals exist in site.goals)
3. Revenue metrics are already dropped in prop breakdown without a goal
   filter via (get_revenue_tracking_currency/3)

* Make it possible to optionally query conversion_rate (continuation)

... also from a custom prop and goal breakdown

* Frontend adjustments to the Locations report

* Display conversion rate in Regions and Cities (ListReport view)
* Display total conversions, conversions (visitors), and CR in the
  "Details" modals of Countries, Regions, and Cities
* Move the percentage into a separate column in the Countries details table

* confirm Stats API breakdown supports conversion_rate (tests only)

* small refactor: extract maybe_add_time_on_page function

* Make it possible to query cr alone

... (without the visitors metric). Already supported in aggregate, this
commit only implements it for the breakdown API.

* Reuse Stats.Util helper functions from b02db88 for aggregate API

We can follow the same logic as with breakdown for manually adding
`visitors` into the metrics list and taking it out of the response
later on.

That way we don't have to make another query, e.g. in a case where
only pageviews and conversion rate is queried. Also keeps things
consistent.

* changelog update

* fix test after resolving merge conflict

* Use explicit string->atom mapping instead of casting

* alias Util module instead of importing it

* use Enum.empty instead of Enum.any

* improve readability

* rename special_metrics to computed_metrics and explain with a comment

* rename visitors_without_event_filters to total_visitors

* keep a single function for removing unwanted metrics

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Adrian Gruntkowski f8b4d5066a
Add multiple imports per site (#3724)
* Clean up references to no longer active `google_analytics_imports` Oban queue

* Stub CSV importer

* Add SiteImport schema

* Rename `Plausible.Imported` module file to match module name

* Add `import_id` column to `Imported.*` CH schemas

* Implement Importer behavior and manage imports state using new entities

* Implement importer callbacks and maintain site.imported_data for UA

* Keep imports in sync when forgetting all imports

* Scope imported data queries to completed import IDs

* Mark newly imported data with respective import ID

* Clean up Importer implementation a bit

* Test querying legacy and new imported data

* Send Oban notifications on import worker failure too

* Fix checking for forgettable imports and remove redundant function

* Fix UA integration test

* Change site import source to atom enum and add source label

* Add typespecs and reduce repetition in `Plausible.Imported`

* Improve documentation and typespecs

* Add test for purging particular import

* Switch email notification templates depending on import source

* Document running import synchronously

* Fix UA importer args parsing and ensure it's covered by tests

* Clear `site.stats_start_date` on complete import to force recalculation

* Test Oban notifications (h/t @ruslandoga)

* Purge stats on import failure right away to reduce a chance of leaving debris behind

* Fix typos

Co-authored-by: hq1 <hq@mtod.org>

* Fix another typo

* Refactor fetching earliest import and earliest stats start date

* Use `Date.after?` instead of `Timex.after?`

* Cache import data in site virtual fields and limit queried imports to 5

* Ensure always current `stats_start_date` is used

* Work around broken typespec in Timex

* Make `SiteController.forget_imported` action idempotent

* Discard irrecoverably failed import tasks

* Use macros for site import statuses

There's also a fix ensuring only complete imports are considered
where relevant - couldn't isolate it as it was in a common hunk

* Use `import_id` as worker job uniqueness criterion

* Do not load imported stats data in plugins API context

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
Karl-Aksel Puulmann f5e15941de
Add migration for minmax indexes on session timestamps (#3772) 3 years ago
Karl-Aksel Puulmann f3509f2a17
Refactor spike detection top sources query (#3770)
* ORDER BY referrer_source for spikes job

This is more consistent with the rest of the queries

* Refactor top_sources -> top_sources_for_spike

* Remove more dead code

* Remove unused arguments

* Remove unused select arguments

* Add a test to top_sources_for_spike
3 years ago
Karl-Aksel Puulmann d1fe184cb7
Remove ORDER BY min(session.start) (#3771)
This adds a needless performance overhead. Note that all queries already
sort by the _name_ of the breakdown value besides the session start.
3 years ago
hq1 99fe03701e
IP Block List (#3761)
* Add Ecto.Network dependency

* Migration: Add ip block list table

* If Cachex errors out, mark the cache as not ready

* Add IPRule schema

* Seed IPRules

* Add Shields context module

* Implement IPRuleCache

* Start IPRuleCache

* Drop blocklisted IPs on ingestion

* Cosmetic rename

* Add settings sidebar item

* Consider IPRuleCache readiness on health checks

* Fix typo

* Implement IP blocklist live view

* Update moduledocs

* Extend contextual module tests

* Convert IPRules LiveView into LiveComponent

* Keep live flashes on the tabs view

* Update changelog

* Format

* Credo

* Remove garbage

* Update drop reason typespecs

* Update typespecs for cache keys

* Keep track of who added a rule and when

* Test if adding via LV prefills the updated_by tooltip

* Update ecto_network dependency

* s/updated_by/added_by

* s/drop_blocklist_ip/drop_shield_rule_ip

* Add docs link

* s/Updated/Added
3 years ago
Karl-Aksel Puulmann 1cb7982cd9
Filtering by multiple custom properties (#3719)
* WIP: PropFilterRow

* Get multi-behavior working

* Render multiple prop filters in one

* Modal reads from query string correctly

* Backend support for multiple custom property filters

* Add backend tests for multiple custom property filters

* Disable already selected options in property keys

We can't allow choosing the same property multiple times without changing the request
params, which we decided against

* Allow choosing any property under Behaviors > Custom props even if custom prop filter applied

This was a limitation (I believe) introduced by using ARRAY JOINs to query custom properties

* CHANGELOG.md

* Solve credo warning about too deep nesting

* Update assets/js/dashboard/stats/modals/prop-filter-modal.js

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

* Refactor internal function for clarity

* Add another step -> Add another

* Solve 500 error

* Separate boxes per property filter

* Retain other filters in props table

* removeFilter behavior for props

* matches_member support for custom props

* filter_suggestions for prop keys should account for prop filter

* find over filter

* refactor appliedFilters

* FILTER_TYPES => FILTER_OPERATIONS

* Make add another link not wrap the whole page

* Unique keys

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
Adrian Gruntkowski 3738cd9578
Parse referrer filter value when passed separately (#3742)
* Parse referrer filter value when passed separately

* Remove unnecessary test setup
3 years ago
hq1 9fb4ea0c3e
Bugfix: available features for expired trials, no subscriptions (#3740)
* Reorganize how subscriptions/trials are evaluated

* Bugfix: expired trial+no subscriptions should not have access to extra features

* Make self-hosted users always on trial

* Seed secondary user with password

* Format

* Fix docs

* Fix small_test run

* Run the test only on full_build

* More tweaks to small builds

* Allow [Goals] for expired trials with no subscription
3 years ago
hq1 931161f693
Generalize Cache/Warmers - extract only specifics to `Site.Cache` (#3716)
* Simplify caches config

The intervals were proven to never change really,
and previous shape made them not very discoverable either.
We'll look into a more declarative setup.

* Aim for more declarative supervision tree setup

* Generalize `Plausible.Cache`

This makes the `Site.Cache` module implement the specifics,
leaving the common bits reusable for upcoming cache processes.

* Generalize `Plausible.Cache` warmers

* Fix typos
3 years ago
hq1 18ddf1e7a0
Re enable accept traffic until notifications (#3737)
* Revert "Remove 0-sized queue"

This reverts commit 6d229b3f70.

* Revert "Temporarily remove notification cron worker (#3732)"

This reverts commit 447e3929cb.

* Revert "Temporarily disable accept_traffic_until actions (#3730)"

This reverts commit 58afe3376f.

* Send accept_traffic_until notifications at 8 UTC rather than 5

* Add option for dry-run

* Expose dry_run in a dedicated function
3 years ago
RobertJoonas 04b9067591
Add `event:goal` property to Stats API (#3718)
* add tests for filtering by goal in timeseries and aggregate

* refactor filter parsing

* stop returning custom props in event:goal breakdown

* test breaking down wildcard pageview goals

* extract filter utils

* parse more goal filter options

* add passing tests for new filter types

* do not allow querying session metrics with a goal filter

* remove unused page_match property

* test that non-configured goals are not returned in breakdown

* enforce filtered goals configured

* update changelog

* Allow simple filtering by revenue goals

This does not mean that revenue metrics are supported. If a revenue goal
is filtered by, we treat it like a simple custom event goal in the API.

* use List.wrap
3 years ago
hq1 712f60f37f
Update `accept_traffic_until` on payment succeeded notification (#3731)
* Update `accept_traffic_until` on payment succeeded notification

* full build only
3 years ago
hq1 58afe3376f
Temporarily disable accept_traffic_until actions (#3730)
* Temporarily disable accept_traffic_until actions

* Remove commented code
3 years ago
hq1 0fa7f3c1e1
Make subsequent query boundaries idempotent (#3723)
* Make subsequent query boundaries idempotent

This should prevent time-sensitive race conditions
on subsequent `Plausible.Stats.*` calls for the same query.
Not sure if there are more places where now() is
dynamic relative to query life cycle, but it's a start.

* Format

* Update lib/plausible/stats/query.ex

Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>

* Match on `Query.now` in a test

---------

Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>
3 years ago
Adrian Gruntkowski 822483c37c
Extract Universal Analytics import logic (#3700)
* Move imported tables schemas to separate modules outside Google ns

* Move buffer for imports to Imported ns

* fix schema newlines

* Extract UA import processing and persistence

* Decouple analytics worker implementation from UA

* Rename env variable for import buffer size

* Preserve old import queue until release
3 years ago
Karl-Aksel Puulmann 11c5d3b251
Remove ARRAY JOINs (#3708)
* Use array indexing instead of left joins

This has multiple benefits:
- Allows us to more easily add support for multiple custom property
  filters
- Performance: Increased performance of some queries by ~20%

Work left to do:
- Check tests coverage
- Create macros for repeated fragments
- Quote and refer to right tables in fragments/macros

* Use macros to clean up code

* Use existing atoms to avoid memory leaks
3 years ago
RobertJoonas 041b0ab779
trace aggregate and timeseries queries too (#3712) 3 years ago
RobertJoonas 6ca4eaf5e2
Improve query tracing (#3704)
* Enable breakdown property tracing for event:goal

This is currently overwritten by event:name. The queries for event:name
will still be tracked if queried specifically.

* Add filters tracing to Query.trace/1

* make sure breakdown metrics are sorted and joined

* check opts skip tracing directly without passing it
3 years ago
Karl-Aksel Puulmann 1b95433d6e
Support `contains`/`matches` for custom properties (#3687)
* Allow `matches` operator to work in BE for custom props

Note: No FE support yet, needs further testing

* feat: allow choosing `contains` for property filters in the UI

* no autocomplete on prop values if `contains` for consistency

* CHANGELOG.md

* Fix: Handle (none) property in property breakdowns when using matching

When matching we should always exclude (none)
3 years ago
Uku Taht a5e0619053
Escape special regex characters (#3634)
* Escape special regex characters

* Add changelog entry
3 years ago
RobertJoonas d3094ffdb7
Prevent upgrades that would leave the customer locked (#3683)
* Remove allowance_required field from grace_period

Since we are now preventing customers from subscribing to a plan that
does not accommodate their pageview usage, there is no need for an extra
check on removing the grace period after a successful upgrade.

This extra check is the reason why the automatic unlocks have recently
failed in several cases.

* refactor outgrown subscription notices

* make a test actually test the described functionality

* Apply greater pageview allowance margin only for trial upgrades

...in order to prevent cancelled or paused subscriptions from subscribing
to plans that would still leave their account locked.

* Mark the entire ChoosePlanTest module full build only

* remove account locking guide

This is irrelevant for self-hosters, and the internal knowledge base is
a better place for this document. Moved it there.

* refactor Keyword get clause

* add a pattern matching assertion in code
3 years ago
RobertJoonas 7536f0285c
Fix weekly interval bug (#3686)
* Fix a bug

This commit fixes a bug where the timeseries weekly interval query for a
month returned 0 visitors for the first week.

This happened because the start of the month defining the first week was
missing the UTC -> site.timezone conversion.

* Update test/plausible_web/controllers/api/stats_controller/main_graph_test.exs

Remove dot from test description

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

---------

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
3 years ago
Adrian Gruntkowski 954b70c79b
Run SiteLocker right after site ownership transfer (#3684)
* Run SiteLocker right after site ownership transfer

* Put site unlocking behind full build flag

* Tag regression tests as full build only
3 years ago
hq1 32456d4348
Upgrade stack (deps, elixir 1.16, OTP 26.2.1) (#3678)
* Bump asdf erlang & elixir

* Bump erlang/elixir in the dockerfile

* Remove Oban.Stager config as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.14.md

* Configure Oban for tests as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.14.md

* Mark sampling hint clause with unsafe fragment

Any other/recommended way to do that? cc @ruslandoga

* Address String.slice/2 deprecation

* Update deps

* Address 0.0 matching warning

* Make funnel settings work

although this is probably not the best way to do it.
Needs revisiting, not sure what broke it - could not find
any breaking changes in related dependencies.

* Migrate oban as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.17.md

* Update credo

* Bump CI cache

* Use `Bypass.pass` to avoid exit shutdown message
3 years ago
RobertJoonas 403f559b35
Limit Custom Properties access in Stats API (#3670)
* add new function and doc for allowed_props

* limit props access in Stats API queries

* use dot syntax instead
3 years ago
hq1 5c8f39ac4c
Prevent crashes on ambiguity when casting timezones (#3663)
* Fall back to UTC when timezone gap found

* bugfix 1

* bugfix 2

* Apply safe tz conversion to all cases where it's done on a ts other than now

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
hq1 72b4e05bbf
Always enable e-mail verification on full build (#3666)
* Always enable e-mail verification on full build

* s/change/set

* Update lib/plausible/auth/user.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Vinicius Brasil 9e5d63ed96
Display premium features tabs on the dashboard (#3646)
* Display premium features tabs on the dashboard

This commit makes the funnels and props tab on the dashboard visible
even when the site owner's plan does not have access to that feature.
This is to raise awareness of those features, and if the site owner does
not want to see that tab, they can still click "Hide this report" to
hide it.

Previously, when the plan did not support a feature, the feature module
`enabled/1` function returned `false`, regardless of the
`sites.feature_enabled` toggle. This commit creates a new function
called `opted_out/1` to differentiate access from explicitly opting out
a feature.

* Remove unused data-conversions-enabled attribute

* DRY Plausible.Billing.Feature.check_availability/1 function

* Allow opting out features the user doesn't have access to

* add upgrade CTA to FeatureSetupNotice

* fix JS linting errors

* simplify notice.js

* fix behaviour when deleting funnels saved to localstorage

In case some other funnel exists, we will use that as the default
selected one. If not, a feature setup notice will be displayed again.

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
3 years ago
RobertJoonas 155a7a56a7
Fix goal filter inconsistency (#3658)
* remove unused code

* add e.name == "pageview" condition to pageview goals

This fixes the weird behavior where filtering by a pageview goal would
also return custom events and vice versa.

* update changelog
3 years ago
hq1 21bbd3835a
Disallow funny business on timezone entry (#3662)
* Disallow funny business on timezone entry

* Add external API test
3 years ago
hq1 8be9397199
Skip breakdown of imported pages with pageviews=0 (#3655)
* Skip breakdown of imported pages with pageviews=0

* Fixup

* Prove good impoted records are merged
3 years ago
hq1 f755b20569
Lock traffic notifications (#3641)
* Update communication

* Remove an unreachable function (mistyped)

* [migration] Make accept_traffic_until a date

* Fix typo

* Set `accept_traffic_until` when creating a site

* Update sites `accept_traffic_until` on subscription change

* Add a note to yearly cancellation notification

* Rephrase annual e-mail for clarity

* Pass the small build test

* Add email notifications

* Fixup

* Implement `accept_traffic_until` notification worker

* Fixup - no need to test this for small build

* Update moduledoc

* Move moduletag

* s/sent_at/sent_on

* Use WHERE NOT EXISTS instead of LEFT JOIN

* Use upsert when tracking notifications sent

* Store sent marker before actually sending notification

* Prefer to keep `accept_traffic_until` on the user record

This gives us a single source of truth, addresses cases like
ownership transparently, simplifies the code and enables CRM toggles.
The only downside is that there's another join performed in the
Sites.Cache full refresh - in this case, small refreshes are
skipped - but this is fine, since the traffic will be let in
anyway.

* Expose `accepted_traffic_until` in the CRM

* Update lib/plausible/auth/user.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Preload owner in CRM

* Use the offset parameter in trial over e-mail contents

* Format

* Harden cache test

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Adrian Gruntkowski 66d4b3a966
Fix SiteLocker for case of user without trial expiry date but with active subscription (#3652) 3 years ago
RobertJoonas d2270f3c35
Small bugfix + refactor email reports (#3642)
* use more convenient testing functions

* do not display + sign with 0% change in emails

* Rename module/file/function names

before, `weekly_report` was also used for monthly reports and that was a
bit confusing to read in code.

* Refactor send_email_report.ex

This commit improves readability by refactoring the code into smaller
functions and reducing the number of arguments given to functions.

But more importantly, it stops making duplicate stats queries for every
email recipient by moving the queries out of the for loop.

* Refactor: move querying logic out of the worker module

and merge all stats information under a single `stats` assign.
3 years ago
Cenk Kücük 147597cd98
Drop events when classified as datacenter IP (#3647) 3 years ago
Adrian Gruntkowski 9d97dc1912
Move limit enforcement to accepting site ownership transfer (#3612)
* Move limit enforcement to accepting site ownerhsip transfer

* enforce pageview limit on ownership transfer accept

* Refactor plan limit check logic

* Extract `ensure_can_take_ownership` to `Invitations` context and refactor

* Improve styling of exceeded limits notice in invitation dialog and disable button

* styling improvements to notice

* make transfer_ownership return transfer to self error

* do not allow transferring to user without active subscription WIP

* Add missing typespec and improve existing ones

* Fix formatting

* Explicitly label direct match on function argument for clarity

* Slightly refactor `CreateInvitation.bulk_transfer_ownership_direct`

* Exclude quota enforcement tests from small build test suite

* Remove unused return type from `invite_error()` union type

* Do not block plan upgrade when there's pending ownership transfer

* Don't block and only warn about missing features on transfer

* Remove `x-init` attribute used for debugging

* Add tests for `Quota.monthly_pageview_usage/2`

* Test and improve site admin ownership transfer actions

* Extend tests for `AcceptInvitation.transfer_ownership`

* Test transfer ownership controller level accept action error cases

* Test choosing plan by user without sites but with a pending ownership transfer

* Test invitation x-data in sites LV

* Remove sitelocker trigger in invitation acceptance code and simplify logic

* Add Quota test for `user.allow_next_upgrade_override` being set

* ignore pageview limit only when subscribing to plan

* Use sandbox Paddle instance for staging

* Use sandbox paddle key for staging and dev

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
3 years ago
RobertJoonas 22ecbe7bc7
Refactor: Split up the choose_plan LV code (#3637)
* move format_price to Plausible.Billing

* move PlausibleWeb.Components.Billing file to subfolder

* extract new Notice module

* rename test file and module name

* move growth_grandfathered notice to notice.ex

* extract a PlanBenefits module

* extract PlanBox component

* extract PageviewSlider component

* fix plan benefits text color
3 years ago
Vinicius Brasil 7f51928338
Remove business tier feature flag (#3632)
* Remove business tier feature flag

This commit removes all code branches related to the business tier
feature flag, as we're not flipping this flag off anymore. It also
removes unused routes, e.g. /billing/change-plan and /billing/upgrade

* remove unused billing templates

* refactor with clause to case instead

* assert on the url in email tests

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
3 years ago
ruslandoga 6639d6af63
Flush Clickhouse buffers faster by encoding early (#3623)
* encode early

* rename: CLICKHOUSE_MAX_BUFFER_SIZE -> CLICKHOUSE_MAX_BUFFER_SIZE_BYTES

* deprecate CLICKHOUSE_MAX_BUFFER_SIZE

* cleanup

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
hq1 8d85c38047
Add `:payment_required` policy to `GateKeeper` (#3628)
* Migration: Add `accept_traffic_until` to "sites"

* Drop traffic from sites where `accept_traffic_until` is passed
3 years ago
RobertJoonas 686ffa6ef3
Clean up legacy trials code + displaying limits (#3620)
* remove unused functionality

We can now safely delete all the logic around users who are on trial and
signed up *before* the business tiers release, since that's no longer
possible

* add monthly_pageview_limit fn clause that takes a user

* make Quota.site_limit return enterprise site limit

...not `:unlimited`, as we still need to display it
in the account settings.

* make `team_member_limit/1` return :unlimited on small_build

* improve team_member_usage/1 function doc

* stop displaying unlimited symbol in usage section

These unlimited limits include:

* `monthly_pageview_limit` for trials
* `team_member_limit` for old enterprise plans
* `site limit` for old accounts (before 2021-05-05)

* format

* small refactor case clause + move mod vars

* review suggestions
3 years ago
hq1 8ba851d27f
Protect against custom props payloads breaking event insertions (#3617)
* Protect against custom props payloads breaking event insertions

* Use two traversals instead of three

* Reorganize invalid props filtering
3 years ago
hq1 b96319321c
Add Browser/Version breakdown to CSV (#3599)
* Add csv for browser versions

* add changelog

* Fixup - ensure the dashboard browser version breakdown still works

* Update CHANGELOG

---------

Co-authored-by: Ekaterina Krivich <ekaterinak@heathmont.net>
3 years ago
Vinicius Brasil 45a763ab2b
Add data retention to the choose plan page (#3605)
* Add data retention field to plans

* Display data retention as benefit when choosing plan

* Split fields in two module attributes

* Remove extra whitespace

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
hq1 7b5c20db17
Revert "experiment in improving throughput of write buffers (#3511)" (#3614)
This reverts commit 16bfb11701.
3 years ago
Adam Rutkowski 69579272d6 Revert "dump buffer on invalid insert (#3613)"
This reverts commit 7a4cafac16.
3 years ago
ruslandoga 7a4cafac16
dump buffer on invalid insert (#3613)
* dump buffer on invalid insert

* dump to PERSISTENT_CACHE_DIR

* fix warning

* ensure dir is not nil
3 years ago
ruslandoga 16bfb11701
experiment in improving throughput of write buffers (#3511)
* wip

* fewer length/1 calls

* encode early

* cleanup

* add micro benchmark

* extract types at compilation

* remove specs to make credo happier

* cleanup is_bounce

* cleanup :D

* add eprof to buffer insert+flush benchmark

* add ci/bench.yml

* rm state.schema from write buffer

* rm noisy logs

* rm benches

* add :source to insert

* make CLICKHOUSE_MAX_BUFFER_SIZE mean bytes

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
Vinicius Brasil e4230db2d9
Improve subscription status checking code (#3598)
* Improve subscription status checking code

This commit improves the subscription status checking in code, and
creates convinience functions to work with it, including nil-checking
and in?/2 function.

* Change in?/2 to macro
3 years ago
Adrian Gruntkowski 12f786810c
Remove `two_factor` feature flag (#3597)
* Remove `two_factor` feature flag

* Avoid compilation warnings in small build
3 years ago
ruslandoga 0eedf9aa98
rm Hammer (#3571) 3 years ago
Marko Saric 0b00762591
Changes to the emails as discussed (#3540)
* Update over_limit.html.eex

* Update dashboard_locked.html.eex

* Update dashboard_locked.html.eex

* Update over_limit.html.eex

* Update dashboard_locked.html.eex

* fix tests

* stop querying owned_site_ids three times

... when querying for billing cycles. Adds an optional `owned_site_ids`
argument to the `usage_cycle` function.

* add penultimate billing cycle info to emails

This commit also refactors some code and adds unit tests to email templates

* use delimit_integer instead of large_number_format

... to display usage with exact numbers such as 1,099,999 instead of 1M

* add penultimate cycle date ranges and linebreaks

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
3 years ago
hq1 615b6aef7d
Plugins API exentsions (custom props, bulk goal delete, goal creation => ListResponse always) (#3593)
* End polymorphic response in goals create

Being part of the v3 spec, this isn't well
(or at all) supported by OpenAPI generators.

Always respond with `Goal.ListResponse`

* Implement `PUT /custom_props`

* Implement bulk `DELETE /goals`

* Expose API for (bulk-)disabling custom props

* Add controller typespecs

* Delegate list wrapping to `Plausible.API.*`
3 years ago
Adrian Gruntkowski da0fa6c355
Implement UI for 2FA setup and verification (#3541)
* Add 2FA actions to `AuthController`

* Hook up new `AuthController` actions to router

* Add `qr_code` to project dependencies

* Implement generic `qr_code` component rendering SVG QR code from text

* Implement enabled and disabled 2FA setting state in user settings view

* Implement view for initiating 2FA setup

* Implement view for verifying 2FA setup

* Implement view for rendering generated 2FA recovery codes

* Implement view for verifying 2FA code

* Implement view for verifying 2FA recovery code

* Improve `input_with_clipboard` component

* Improve view for initiating 2FA setup

* Improve verify 2FA setup view

* Implement `verify_2fa_input` component

* Improve view for verifying 2FA setup

* Improve view rendering generated 2FA recovery codes

* Use `verify_2fa_input` component in verify 2FA view

* Do not render PA contact on self-hosted instances

* Improve flash message phrasing on generated recovery codes

* Add byline with a warning to disable 2FA modal

* Extract modal to component and move 2FA components to dedicated module

* First pass on loading state for "generate new codes"

* Adjust modal button logic

* Fix button in verify_2fa_input component

* Use button component in activate view

* Implement wait states for recovery code related actions properly

* Apply rate limiting to 2FA verification

* Log failed 2FA code input attempts

* Add ability to trust device and skip 2FA for 30 days

* Improve styling in dark mode

* Fix waiting state under Chrome and Safari

* Delete trust cookie when disabling 2FA

* Put 2FA behind a feature flag

* Extract 2FA cookie deletion

* ff fixup

* Improve session management during 2FA login

* Extract part of 2FA controller logic to a separate module and clean up a bit

* Clear 2FA user session when rate limit hit

* Add id to form in verify 2FA setup view

* Add controller tests for 2FA actions and login action

* Update CHANGELOG.md

* Use `full_build?()` instead of `@is_selfhost` removed after rebase

* Update `Auth.TOTP` moduledoc

* Add TOTP token management and make `TOTP.enable` more test-friendly

* Use TOTP token for device trust feature

* Use zero-deps `eqrcode` instead of deps-heavy `qr_code`

* Improve flash messages copy

Co-authored-by: hq1 <hq@mtod.org>

* Make one more copy improvement

Co-authored-by: hq1 <hq@mtod.org>

* Fix copy in remaining spots

* Change redirect after login to accept URLs from #3560 (h/t @aerosol)

* Add tests checking handling login_dest on login and 2FA verification

* Fix regression in email activation form submit button behavior

* Rename `PlausibleWeb.TwoFactor` -> `PlausibleWeb.TwoFactor.Session`

* Move `qr_code` component under `Components.TwoFactor`

* Set domain and secure options for new cookies

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
RobertJoonas 4566e6b530
New admin route for displaying usage (#3577)
* add a new crm usage route for admins

* add a test for admin route authorization

* add full_build_only tag
3 years ago
Uku Taht 032823e112
Add last_bill_date to new subscriptions (#3588)
* Add last_bill_date to new subscriptions

* Remove leftover test code

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
hq1 7bf1e2a6ed
Reapply "Define a better monthly pageview usage (#3564)" (#3574)
This reverts commit c739b8878d.
3 years ago
hq1 5278c23965
Revert and extract migration (#3573)
* Revert "Define a better monthly pageview usage (#3564)"

This reverts commit 57188a402a.

* Extract migration from 57188a402a/priv/repo/migrations/20231129103158_add_allow_next_upgrade_override_to_users.exs
3 years ago
RobertJoonas 57188a402a
Define a better monthly pageview usage (#3564)
* refactor asking for the monthly pageview usage

* add tests for usage and limits section in account settings

* display pageview usage per billing cycle for active subscribers

* disable cycle tabs if no usage

* make current billing cycle whole

...instead of capping it at today's date

* run queries for different cycles concurrently

* fix linebreak bug

* add calculate usage action into CRM

* change some names of assigns

* block subscribing to a plan by pageview usage

Depending on whether the customer has already subscribed or not, checking
their pageview usage is different:

* If they're not subscribed yet, we allow them to subscribe to a plan If
  it their last 30 days usage does not exceed the plan pageview limit by
  more than 15% (30% for when subscribing to a 10k plan)

* For existing subscribers, we'll use the exact same mechanism that we're
  using for locking sites - the last two billing cycles usage. If both
  cycles exceed the plan limit by more than 10% - we don't allow them to
  subscribe to the plan

* apply credo suggestion

* prevent highlight bar overflow

* move disabled classes to button element

* optimize for darkmode

* unify link and text styling on the same horizontal line

'Upgrade' & 'Update billing details' links + billing interval text were
positioned on the same line. The font size was similar, but not the same

* improve exceeded_limits function readability

* Refactor some tests and remove code duplication

* override allow upgrade when limits exceeded

In cases where limits are exceeded, we can set the boolean flag
`allow_next_upgrade_override` to `true` in the CRM. This will allow
the user to upgrade to any plan they want. After they've upgraded or
changed their plan - the flag will automatically reset to `false`.

* only apply upgrade override for exceeded pageview limit

* fix tests on the CI

* make current_cycle usage always displayed by default

* make pageview allowance margin more clear

* add comment
3 years ago
hq1 a4b9c3b8ba
Remove custom domains support + update build options (#3559)
* Disable super-admin checks on small build

* Mute a test writing to stdout

* Move sampling outside of small build

* Convert waiting_first_pageview to heex and stop relying on env vars

* Set site limit unlimited on small build

* Stop relying on app env to get trial expiry

* Remove custom domains - including migration

* Remove is_selfhosted from layout view

* Quota fixup

* Stop relying on app env for self hosted registration

* Stop relying on app env for pass reset success

* Apply on_trial? check only on full build

* Update templates relying on app env

* Adjusts auth controller tests for small build

* Trial fixup

* Fixup

* Stop relying on app env

* Rest of the fsckn owl

* Update typespecs

* Fix dialyzer warning

* Remove unused module

* Credo + format

* GeoIP is not, for full build

* Use `small_build?()` where applicable

* Implement bypassing FirstLaunchPlug without insertions

* Get Marko's patch de58a18a85

* Test is-dbip=false presence

* Fix typespec

* Remove future hardcodes

* Handle `nil` from `Plausible.Geo.database_type()`

* Remove XXX marker

* Use one typespec for two clauses

* Introduce `MIX_ENV=small_dev`

* Revert "Use one typespec for two clauses"

This reverts commit 8d8cd21764.
3 years ago
Adrian Gruntkowski 86b9d1680d
Simplify TOTP core logic and add email notifications (#3563)
* Modify API of `Auth.TOTP` to work with simplified flow

* Send email when 2FA is enabled and disabled

* Add tests for `initiated?/1`

* Add tests for email submission and improve recipient address composition

* Fix email tags

* Rename email templates to HEEx

* Fix formatting in email templates
3 years ago
RobertJoonas ff2c3346d2
Bugfix: Allow breakdown by internally used prop keys for Growth plans (#3562)
* fix bug - allow internally used prop key breakdown for Growth plans

* use case instead of with
3 years ago
hq1 0f577a998c
Update credo config + extras (#3547)
* Disable CyclomaticComplexity checks

* Move AuhtorizeSitesAPI plug under extra
3 years ago
hq1 88e1d9dc28
Small build updates (#3546)
* Sites API

* Extract Revenue react api helpers

* !fixup

* Extract JS Money module to /extra

* Extract Revenue full build extras (tests pass for full)

* Update MIX_ENV=small mix test

* Remove dead code

* Add moduledocs

* Add credo config

* Trick dialyzer

* DRY revenue metrics

* Use more concise version of on_full_build macro

* Disable credo check
3 years ago
Vinicius Brasil b35096bbc8
Dump plan information to PostgreSQL (#3543)
* Use Ecto.Schema for casting plans from JSON files

* Dump plans to internal database table
3 years ago
Adrian Gruntkowski 65cc8980e0
Implement core logic for TOTP support (#3525)
* Add `nimble_totp`, `cloak` and `cloak_ecto` to project dependencies

* Setup Cloak-based secrets vault and create a dedicated Ecto type

* Add `totp_enabled|secret|last_used_at` fields to `User` schema

* Implement schema and stateless logic for TOTP recovery codes

* Implement core logic of TOTP auth

* Fix typos and improve style of doc comments

Co-authored-by: hq1 <hq@mtod.org>

* Fix moduledoc alignment

* Use more compact conditional expression

Co-authored-by: hq1 <hq@mtod.org>

* Disambiguate `I` as `7` when generating recovery codes (h/t @hq1)

* Fix a typo in runtime config error message

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
hq1 b9ec38038c
Add small build option (#3536)
* Update applications

* Clone community config

* Move modules to experimental dir

* Update runtime config

* Apply first set of compile-time conditionals

* Move funnel schemas to experimental

* Make funnel schema-less build compile

* Use experimental/lib for elixir code

* Move JS funnels to experimental

* Clean up conditional rendering

* Tidy up the pipeline

* Make two builds pass tests without warnings

* Reuse existing dotenvs

* Do a bunch of renames

* Clean up naming

* Run secondary CI

* Update router

* Remove RewriteFunnelDupes migration

Tests were disabled already and it was a one-off shot

* Fixup quota mixins

* Add moduledoc

* Change MIX_ENV for seconary test run

* Skip crm on small

* !fixup

* Exclude flags pipeline

* Update lib/plausible_web/controllers/stats_controller.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
RobertJoonas 02a1271ee4
Bugfix site transfers (#3531)
* Bugfix: allow ownership transfers when premium features enabled but not used

Fields like `props_enabled` and `funnels_enabled` are true by default,
and these fields do not indicate whether the user/site is actually using
these features or not.

* allow site transfers if they will be at limit after transfer

* small refactor
3 years ago
Vinicius Brasil 0175158e81
List only premium features on the CRM (#3530) 3 years ago
Cenk Kücük f4d7ba70e8
drop IPs classified as datacenter (#3522)
* drop IPs classified as datacenter

* process event and increment counter instead of dropping
3 years ago
Márton Salomváry 555eb25d20
Allow custom event timeseries in stats API (#3505)
* Allow custom event timeseries in stats API

* Fix linting error

---------

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
3 years ago
Vinicius Brasil cfaa5be8f4
Configurable limits for enterprise plans (#3527) 3 years ago
RobertJoonas d66322e12d
Limit grandfathering to *active* subscribers and trials (#3524)
* refactor asserting plan generation in plans_test.exs

* stop grandfathering old expired trials

For users who registered before the business tiers release, we want to
offer a chance to subscribe to a grandfathered plan. However, if they
let their trial expire and don't subscribe in the next 10 days, they'll
lose that opportunity.

* stop grandfathering expired subscriptions

* remove default title and icon from Generic.notice

* fix bug with dismissable notice

classList is null when dismissable_id is not given

* alias Plausible.Auth.User

* Refactor Generic.notice component

Make it easy to apply different colors

* move subscription_cancelled_notice across the app

And remove from user settings > subscription box. Also, include a note
about losing grandfathered status when letting the subscription expire.

* allow full width in Generic.notice

* use Generic.notice for subscription_past_due_notice

* use Generic.notice for subscription_paused_notice

* prevent two notices clashing into each other with gap-y-2

* define attrs for phx components

* optimize for light mode

* make subscription cancelled notice dismissable

but if it's dismiss, show it in the place where it was before in the
account settings > subscription box

* make function private

* replace function doc with regular comment to avoid compile warning

* use array for classnames

Co-authored-by: Vinicius Brasil <vini@hey.com>

* fix typos in function doc

---------

Co-authored-by: Vinicius Brasil <vini@hey.com>
3 years ago
ruslandoga 13055aafc0
use windows in 'time on page' queries (#3446)
* use windows in 'time on page' query

* add imported time on page back

* join imported_pages instead of extra query

* don't forget aggregated time on page!

* eh?

* no need for separate window clause

* use dynamic_filter_condition in aggregate_time_on_page

* it's avg, not total

* fix dynamic_filter_condition call

* tests pass

* fewer changes

* fewer changes + feature flag

* base quickfix

* fewer changes

* fewer changes?

* fewer changes!

* replace coalesce with if(empty(),etc)

* make window_aggregate_time_on_page return same result as neighbor_

* breakdown as well

* add failing test

* more info

* more info

* format example

* add failing aggregate.ex test

* fix breakdown.ex windowing time_on_page calculation

* fix aggregate.ex windowing time_on_page calculation
3 years ago
Adrian Gruntkowski 3ba57a04fc
Make OpenTelemetry sampler ratio configurable via env (#3514) 3 years ago
Adrian Gruntkowski f464ceae88
Implement pinned sites (#3469)
* Revert "Remove site pins for now"

This reverts commit 5eccf4eaf6.

* Implement basic site pin schema level logic within user specific preferences

* Add vertical ellipsis menu markup

* Implement basic changesets for user preferences

* Implement pin toggling

* Try to fix pin sorting

* Implement pin toggling in LV

* Adjust moduledocs for new schema(s)

* Remove unnecessary `distinct` from query

* Use `button` for pin/unpin action

* Generalize preference setting

* Rename schema and fields for clarity

* Rename `list_type` -> `entry_type`

* Safeguard setting options

* Test `set_option/4` and `toggle_pin/2`

* Add test for listing pinned sites via `Sites.list`

* Disallow pinning sites outside page explicitly

* Test pinning in LV

* Test conditional rendering of site settings in /sites

* Remove unnecessary TODO comment

* Safeguard `Sites.set_option/4` against invalid user/site combo

* Handle pinned sites in dashboard site picker

* Clear flashes upon (un)pinning sites

* Update CHANGELOG

* Prevent blinking of hamburger menu items on first paint

* Highlight hamburger handle on hover in /sites

* Start showing hotkeys in site picker again

* Sort pinned sites in the order they were pinned

* Update sites list order immediately after pin/unpin toggle

* Refactor and split `Sites.list/3`, extracting `Sites.list_with_invitations/3`

* Cap number of pinned sites at 9 per user

* First pass on visual indication of site cards (dis)appearing

* Apply ellipsis gradient+shadow on card hover

* Fix responsive padding of site cards

* Sort by invitations first, pinned sites second and then the rest

* Revert "Apply ellipsis gradient+shadow on card hover"

This reverts commit 0608796612639030ccbb12df639709f78edc1434.

* Apply more subtle hover effect on the ellipsis menu

* Make error and success flash LV boxes use separate component containers

* Promote `pinned_at` in table migration to a column

* Switch logic to using `pinned_at` as a standard schema field

* Refactor `Sites.list*` getting rid of subquery (h/t @ukutaht)

* Remove migration which is already merged upstream

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
3 years ago
Adrian Gruntkowski 2d2ea1a2a3
Implement trace ID ratio-based sampling for OpenTelemetry (#3499)
* Implement trace ID ratio-based sampling for OpenTelemetry

* Make `decide` more resilient
3 years ago
RobertJoonas 7036332db2
Business tier bug smash (#3493)
* price formatting

* fix space underlined

* add a redirect from :upgrade to :choose_plan

* Add premium feature notice to revenue goals

* add the number of API request to plan benefits

* be more explicit about Stats API requests

* start linking to the new upgrade page if FF enabled

* add redirect to :upgrade_to_enterprise_plan from :choose_plan

This commit intends to keep the existing behaviour of redirecting users
directly to the enterprise upgrade page when they click on an upgrade
link in an email and have an enterprise plan configured.

Hence, we can also simplify the /settings template and only link to
'/billing/choose-plan', even for enterprise plans.

* deprecate /billing/change-plan route based on FF

* Move new site template to HEEX

* Move new team member invitation template to HEEX

* Standardize exceeded limits error message

* Fix failing tests

* Limit Stats API access to unlimited trials

---------

Co-authored-by: Vinicius Brasil <vini@hey.com>
3 years ago
Vini Brasil c9bf5827e9
Prepare business tier for release (#3464)
* Change limits for trials

* Keep legacy trial limits for users that registered before the business tier

* Change private preview notice for release

* Run formatter

* Add countdown to private preview notice
3 years ago
RobertJoonas f977351ae2
Fix upgrade page for legacy trials + bug fixes (#3486)
* allow using Stats API and Props for free_10k subscriptions

* return v3 plans for legacy trials

* do not display grandfathering notice for legacy trials

* set a more accurate BT release date

* fix bug on dev env

Allow the `find/1` function to find sandbox plans

* add error handling and tests for change_plan_preview

* fix feature warning bug

* fix credo warnings

* fix tests

* set BT release date further into the future

* rename function and some vars

* bugfix with limit exceeding

* fix test
3 years ago
hq1 08bd10614f
Fixups sites (#3482)
* Revert "Fix sites query invitation left join lookup (#3480)"

This reverts commit 6e6508a359.

* Revert "Revert "Fix sites query invitation left join lookup (#3480)""

This reverts commit f5cc831d7a.

* Revert "Fix sites query invitation left join lookup (#3480)"

This reverts commit 6e6508a359.

* Fix sites listing
3 years ago
RobertJoonas df44f549d8
Recommending a plan (#3476)
* use a different article in the email copies

... for recommending a plan, since the user can choose between Growth
and Business.

* small refactoring improvement

Rename `Plans.available_plans_with_prices` to `Plans.available_plans_for`,
taking an optional `with_prices` argument.

* highlight recommended tier for trial users on the ugprade page

* review suggestion
3 years ago
hq1 6e6508a359
Fix sites query invitation left join lookup (#3480) 3 years ago
Adrian Gruntkowski 07cab27fef
Implement new sites view (#3463)
* Implement complete basics of LV sites

* Reimplement everything in LV except pagination

* Implement basic search capability

* PoC: plot visitors on sites index

* Add rudimentary clipped gradient in minicharts

* Fix clipping gradient, define once

* Format

* Add moduledoc to visitors component

* Move paginator helpers to the top core namespace

* Fix typespec of `Plausible.Sites.list`

* Split sites component into subcomponents

* Add function to uniformly calculate 24h intervals
and visitor totals across multiple sites.

* Integrate batch 24h interval query with plots on sites view

* Don't confuse heex compiler with alpine @ shorthands

* Make linear gradient svg definition truly invisible

* Implement basic pagination

* Extract `site_stats` from site and invitation cards

* Improve pagination

* Tweak css

* Improve filtering on pagination and make WSS fail graceful

* Test `last_24h_visitors_hourly_intervals/2`

* Replace /sites with LV implementation

* Add debounce to search filter

* Fix typespecs

* Fix styling

* Fix mini graph scaling factor calculation

* Fix search consuming itself

* Minimal tweaks to the plots

* Fixup

* Remove magic numbers from the plot

* Create `site_pins` table

* Add `SitePin` schema

* Implement listing invitations, sites and pins in a single query

* Add FIXME note

* Remove site pins for now

* Add tests for `Plausible.Sites.list/3`

* Add a couple more tests to sites dead view

* Remove unnecessary FIXME

* Add LV tests for Sites

* Calculate and display 24h visitors change

* Render the change in bold

* Add clarfying comment on virtual field in `Site` schema

* Remove unnecessary function from Invitations API

* Remove unused list opt from type definition in `Sites`

* Improve joins in list query slightly

* Add comment on manually computing sites list total

* Start searching from a singly character in domain field

* Add typespec to `last_24h_visitors_hourly_intervals`

* Extend moduledoc in visitors component

* Simplify loading sites in LV

* Simplify assigns in LV

* Add missing group for shadow under site card

* Make invitation modal render

* Make HTML in sites LV semantically correct

* Remove autofocus and focus search on `/`

* Remove shadow from search input

* Make search cancel on escape

* Fix tests relying on outdated HTML structure

* Make visitor chart color scheme consistent with dashboard chart

* Update styling of trend labels

* Fix empty state and improve search blur/focus handling

* Use live navigation for pagination

* Implement spinner on load from search

* Remove unused `Plausible.Stats.Clickhouse.last_24h_visitors/1`

* Calculate uniques correctly across hour boundaries

* Swap inlined svg for Heroicons component in invitation modal

* Add order by to base query in 24h hourly intervals

* Revert "Add order by to base query in 24h hourly intervals"

This reverts commit a6be5e3026.

* Query clickhouse 24h visitors only on second mount

* Remove redundant sign from percentage change when negative

* Switch to offset-based pagination

  - offset seems easier to deal with for when actions on
    paginated list will be performed such as site pinning;
    tracking cursor data makes some entries disappear in
    edge cases. The data set is still fairly small and
    static, even for large customers.
  - we're removing Phoenix.Pagination as it doesn't really
    fir any use case, and it was only used to limit the number
    of sites in the site picker
  - site picker is now limited to 9 sites (future: pinned
    sites will be prioritized there)
  - no need to re-query for total count any more
  - BTW, the old /sites template was removed

* Refine the plot queries; Tests pass snapshot

* Add PromEx plugin for LiveView

* Fix tiny plot cut-off at the top

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
3 years ago
Vini Brasil 3b28a8d418
Warn user about Stats API when downgrading (#3471)
* Warn user about Stats API when downgrading

* Update test/plausible/billing/quota_test.exs

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
RobertJoonas 3c76053c3f
ignore only pv limit exceeded on subscribe (#3470) 3 years ago
RobertJoonas 8cc7bce689
Restrict subscribing to a plan when exceeding its limits + warning for losing feature access (#3461)
* fix the styling of the red text notice under checkout link

* avoid some code repetition

* simplify rendering the change_plan_link

* refactor disabling checkout link and showing disabled message

* disable change plan and upgrade link when exceeding pageview limit

* disable checkout when exceeding team member limit

* disable checkout when site limit exceeded

* extract checkout related code in a separate function

* stick to a single order of features

* losing features warning

* fix back link from change-plan-preview

* create Quota.exceeded_limits function

* restrict subscribing with exceeded limits on the API level too

* use with instead of case

Co-authored-by: Vini Brasil <vini@hey.com>

* use :map type instead of :any for user

Co-authored-by: Vini Brasil <vini@hey.com>

* create Quota.usage function

---------

Co-authored-by: Vini Brasil <vini@hey.com>
3 years ago
Vini Brasil ad05af08a4
Check for limits and features used by site before transferring ownership (#3445)
* Simplify team_member_usage query

* Check limits before transferring ownership

* Extract invite creation to dedicated service module

* Simplify team member usage query

* Remove unnecessary distinct clause

* Delegate CreateInvitation via Memberships
3 years ago
RobertJoonas a226773da1
Legacy plans (#3455)
* replace unlisted plans with legacy plans

...and add a legacy plan that has an existing subscriber in Paddle. All
legacy plans are considered generation 1 - meaning that when a user on
one of these plans and they'll go to the upgrade page, then the listed
plans will be v1 for Growth and v3 for Business.

* remove redundant plans_sandbox function

* remove the unused 'scope' argument from Plans.find

* remove unused plan

* add basic test coverage for legacy plans

* add another plan with an existing active subscriber (fix another bug)
3 years ago
hq1 117eef000d
Upgrade Erlang/Elixir stack (#3454)
* Bump deps

* Bump stack

* Fix deprecation warnings

* Fix VCR cassettes mismatch due to OTP-18414

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Format & fix flaky tests

* Handle raw IPv4 hostnames; test public suffix TLD

* Configure locus db cache_dir

So that maxmind unavailability doesn't affect
application startup. PERSISTENT_CACHE_DIR env var is used
to point locus at the GeoIP DB file.

* WIP: Remove ExVCR

* Fix test env config

* Fixup exvcr

* Remove exvcr from deps

* Add convert script

* Remove exvcr cassettes

* Remove convert script

* Rename test

* Update moduledoc

* Update dockerfile

* Bump CI cache

* Tag more slow tests, why not?

* Use charlist for locus cache option

* Pin nodejs

* Merge google tests, make them async

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
RobertJoonas 2ada3d700f
List plan benefits on the new upgrade page (#3444)
* change team member limits for new v4 plans

* duplicate business plans with unlimited team members

We need to do this because we want grandfathered users to have unlimited
team members on business plans as well. Otherwise we'd have to build
overrides on the subscription level when checking the limit.

* refactor generating plan structs

* move Plan module into a separate file

* remove not needed conditions

* add generation field to plans

* sync the sanbox plan limits and features with plan generations

* implement displaying plan benefits

* add grandfathering notice

* plug in the real v3 business plan IDs

* optimize N/A text color for darkmode

* use String.to_existing_atom instead

Co-authored-by: Vini Brasil <vini@hey.com>

* Remove the unnecessary part of a comment

Co-authored-by: Vini Brasil <vini@hey.com>

* make the Plan.new function simpler

* use exlamation marks

---------

Co-authored-by: Vini Brasil <vini@hey.com>
3 years ago
hq1 957138a8ec
Plugins API: handle Revenue Goals creation wrt Business Tier (#3440)
* Plugins API: handle Revenue Goals creation wrt Business Tier

* Update bulk goal creation

* Update test/plausible_web/plugins/api/controllers/goals_test.exs

Co-authored-by: Vini Brasil <vini@hey.com>

* Use growth_subscription factory

---------

Co-authored-by: Vini Brasil <vini@hey.com>
3 years ago
hq1 19b2239eb9
Track tokens usage (#3438)
* Migration: track last seen usage for Plugins API Tokens

* Track and interpret Token.last_seen_at

* Display last used

* Order tokens by inserted date, rather than UUID :clown:

* s/Last seen/Last used in the UI

* Test for "Last used" column presence

* Fix table layout for very long descriptions

* Update lib/plausible/plugins/api/tokens.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible/plugins/api/token.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update test/plausible/plugins/api/token_test.exs

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* s/last_seen_at/last_used_at

* Update lib/plausible_web/live/plugins/api/settings.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* fixup

* Document reasoning behind 5m windows

* s/last_seen/last_used

* Mute credo

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
hq1 2cc80ebd7a
Integrations Settings section (#3427)
* Extend the Tokens context module

* Extract GA Import to separate component

* Extract Search Console settings to separate component

* Remove Search Console from the router

* Stop counting imported pageviews in general settings

* Remove search console controller action

* Add settings_integrations controller action

* Fix remaining redirects

* Add Integrations route

* Replace SC sidebar item with Integrations

* Update site controller tests

* Implement Plugins API Tokens LV

* Apply universal heroicon to docs info links

* Add flash on token creation

* Update CHANGELOG

* Redirect to integrations upon forgetting GA import

* Update moduledocs

* Remove unnecessary wildcards

* WIP: attempt at fixing broken oauth flow

* Fix post-import redirect

* Fixup missing attribute

* Format

* Seed random google auth

* Use example.com for seeded e-mails

* Tweak Google integrations layout

* Remove dangling IO.inspect

* Bugfix: copy to clipboard breaking LV form bindings

* Update lib/plausible/plugins/api/tokens.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible_web/controllers/site_controller.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible_web/live/plugins/api/settings.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update test/plausible/plugins/api/tokens_test.exs

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago