Compare commits

...

289 Commits

Author SHA1 Message Date
Adrian Gruntkowski a7603c9e49
Improve import procedure to ensure no time range overlaps (#3970)
* Always scope import ID by site as well

* Do not schedule new import job if there are any site imports in progress

* Disable import buttons when any import is in progress

* Simplify `schedule_job/4` (h/t @RobertJoonas)
3 years ago
Adrian Gruntkowski cffff0340c
Handle Google API timeouts gracefully during imports (#3975) 3 years ago
Adrian Gruntkowski 33eed9d7db
Delete imports which have no stats (#3972) 3 years ago
Marko Saric e5b7f1afd0
Changing the copy of the locked screen (#3967)
changing the copy here as I think that in some situations the "we're still counting stats" message is now shown even to those dashboards where we've stopped counting stats so best to avoid that
3 years ago
Uku Taht f966419a4a
Update ua_inspector (#3957)
Co-authored-by: hq1 <hq@mtod.org>
3 years ago
hq1 f9f0407d68
Remove `experimtnal_hostname_filter` and keep it on by default (#3973)
* Remove `experimental_hostname_filter` and keep it on by default

* Catch up with changes done via e5b56dbe6
3 years ago
RobertJoonas e5b56dbe62
Refactor VisitorGraph (#3936)
* Give a more semantic name to a function

* Make the LineGraph component thinner

* Move LineGraph into a separate file

* Move interval logic into interval-picker.js

This commit also fixes a bug where the interval name displayed inside
the picker component flickers the default interval when the graph is
loading.

The problem was that we were counting on graphData for returning us the
current interval: `let currentInterval = graphData?.interval`

We should always know the default interval before making the main-graph
request. Sending graphData to IntervalPicker component does not make
sense anyway.

* extract data fetching functions out of VisitorGraph component

* Return graph_metric key from Top Stats API

This commit introduces no behavioral changes - only starts returning an
additional field, allowing us to avoid the following logic in React:

1. Finding the metric names, given a stat display name. E.g.
   `Unique visitors (last 30 min) -> visitors`

2. Checking if a metric is graphable or not

* Move metric state into localStorage

This commit gets rid of the internal `metric` state in the VisitorGraph
component and starts using localStorage for that instead.

This commit also chains the main-graph request into the top-stats request
callback - meaning that we'll always fetch new graph data after top stats
are updated. And we do it all in a single function.

Doing so simplifies the loading state significantly, and also helps to
make it clear, that at all times, existing top stats are required before
we can fetch the graph. That's because the metric is determined by which
Top stats are returned (for example, we can't be sure whether revenue
metrics will be returned or not).

* Make sure graph tooltip says "Converted Visitors"

* Extract a StatsExport function component

Again, instead of relying on `graphData?.interval` we can read it from
localStorage, or default to the largest interval available. The export
should not be dependant on the graph.

* Extract SamplingNotice function component

* Extract WithImportedSwitch function component

* Stop "lazy-loading" the graph and top stats

Since the container is always on top on the page, it will be visible on
the first render in any case - no matter the screen size.

* Turn VisitorGraph into a function component

* Display empty container until everything has loaded

* Do not display loading spinner on realtime ticks

* Turn Top Stats into a fn component

* fetch top stats and graph async

* Make sure revenue metrics can remain on the graph

* Add an extra check to canMetricBeGraphed

* fix typo

* remove redundant double negation
3 years ago
Karl-Aksel Puulmann 3115c6e7a8
Reducing JOINs in queries (#3966)
* Move experimental_session_count? logic to within query object

* WIP new querying system for deciding what tables to query

* both -> either

* Include sample_percent in both tables

* Remove a hanging TODO

* Allow filtering by visit props on event queries if flag is on

* Make default sessions join more conditional

* Simplify events_join_sessions?

* Add some TODOs

* Fix assignment

* Handle entry/exit page visit props separately from props stored in events table

* Update test which created sessions/events differently from everyone else

* Make query_events private

* Dont filter by session properties on events table if querying sessions and joining in events

* Handle visits, pageviews, events and visitors metrics from other table

* both -> either

* events, pageviews are strictly event metrics

* Add support for (plain) breakdowns deciding which table to use

* Run tests with experimental_reduced_joins as a separate job

Also refactor which tests are run with postgres:15 to reduce number of jobs

* moduledocs for TableDecider

* Fix matrix

* Custom build name

* Move TEST_EXPERIMENTAL_REDUCED_JOINS check

* Handle percentage separately from other metrics

* Remove debug code

* TableDecider tests

* both => sample_percent

* Improve naming

* Simplify code

* Breakdowns retain old behavior if getting metric visitors

* Unify behavior of entry/exit page hostnames with rest

* Fix test naming
3 years ago
hq1 6af80dd246
Filter by hostnames (#3963)
* CH Migration: exit/entry hostnames in sessions_v2

* Leave only exit_page_hostname, we already record hostnames

* Use ClickHouse DDL in favour of ecto so that cluster is included

* Compress with ZSTD(3)

* Expose Hostname filter in the dashboard dropdown

* Add `exit_page_hostname` to ClickHouse `sessions_v2` schema

* Start tracking hostname changes in sessions

* Implement hostname filter suggestions

* Enable filtering by `event:hostname`

* Add tests for filtering by hostnames

* Ensure filter suggestions work for exit pages too

* Allow overriding hostnames with `send_pageview` mix task

* Remove `:window_time_on_page` flag

It seems that we can remove it after all?

* Initialize `experimental_hostname_filter` query parameter

* Rewrite cache store behaviour with regards to session hostnames

* Work around inconsistent session merging

So that `populate_stats` can get closer to actual ingestion

* Improve top stats test

* Make it possible to filter sessions by entry/exit hostnames

* Update pages tests

* Expose `experimental_hostname_filtering` temporarily in the UI

* Untested yet: also apply experimental filtering to sources

* Introduce `hostname_filter` feature flag

* Format

* Test top sources with hostname filter + experimental flag
3 years ago
Adrian Gruntkowski e6d83e946f
Populate new columns in imports and exports (#3969)
* Extend `Imported*` schemas with newly added columns

* Populate newly added `Imported*` fields in GA4 imports

* Extend exports with newly added fields

* Extend CSV importer to ingest new fields

* Fix alias shadowing error

* Add more extensive GA4 import fixtures

* Apply rounding and casting to sampled visits
3 years ago
RobertJoonas bd73fc8266
CH Migration - add more imported metrics and properties (#3949)
* add migration

* add utm_source to imported_sources

* quickfix

* satisfy credo

* Revert "satisfy credo"

This reverts commit bb0b228164.

* Revert "quickfix"

This reverts commit ab6f70c79e.

---------

Co-authored-by: ruslandoga <67764432+ruslandoga@users.noreply.github.com>
3 years ago
ruslandoga 2ba4988c95
specify insert columns in import from s3 (#3968)
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
hq1 1f778e0c11
CH Migration: exit page hostname on sessions_v2 (#3953)
* CH Migration: exit/entry hostnames in sessions_v2

* Leave only exit_page_hostname, we already record hostnames

* Use ClickHouse DDL in favour of ecto so that cluster is included

* Compress with ZSTD(3)
3 years ago
Adrian Gruntkowski 9f27fa303c
Fix dry run mode in `DataMigration.SiteImports` (#3965) 3 years ago
Adrian Gruntkowski 23a3699dd7
Improve import stats toggle and `with_imported` flag computation (#3960)
* Check import presence across all imports and not just the first one

Also, simplify imported data toggle rendering to not explicitly
refer to the earliest import source.

* Change imported stats toggle icon in dashboard

* Test `Imported.get_imports_date_range/1`

* Simplify failed UA/GA import email copy
3 years ago
Adrian Gruntkowski 71fe541359
Implement script for backfilling legacy site import entries and adjusting end dates of site imports (#3954)
* Always select and clear import ID 0 when referring to legacy imports

* Implement script for adding site import entries and adjusting end dates

* Log cases where end date computation is using fallback

* Don't log queries when running the migration to reduce noise
3 years ago
Adrian Gruntkowski 5bf59d1d8a
Implement adjusting imported date range to actual and existing stats (#3943)
* Implement adjusting imported date range to actual and existing stats

* Drop redundant prefix from import list entries

* Make pageview numbers in imports list formatted for readability

* Test and improve date range cropping

* DRY UA and GA4 stats start and end date API calls

* Extend UA/GA import controller tests and improve error handling

* refactor finding longest open range without existing data

* Fix typo in test description

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

* Rename `open_ranges` to `free_ranges`

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
ruslandoga c263df5805
CSV imports (UI) (#3845)
* add basic ui

* remove TODO

* credo

* allow folder upload

* redirect external

* mention folder, use folder icon for file picker

* back to multiple file upload

* mention zip

* escape dots in archive filename
3 years ago
hq1 b31433a7bf
Ensure all the react container attributes are strings (#3948) 3 years ago
hq1 edf70d14b6
Use sessionStorage for "dashboard first launch" banner tracking (#3892)
* Use sessionStorage for offer e-mail report banner tracking

Keeping it within the cookie is problematic, as the banners don't
expire and overflow the cookie with data when enough new sites
are added.

Ref https://github.com/plausible/analytics/issues/3762

* Update changelog

* Extract a component

* Make is_dbip evaluate to quoted boolean
3 years ago
hq1 7523abe93e
Add metrics to ingestion pipeline (#3927)
* Add metrics to ingestion pipeline

* Format

* Format

* Update buckets

* Credo
3 years ago
Karl-Aksel Puulmann 604bf88451
Add github action to validate whether migrations and app change at the same time (#3945) 3 years ago
Karl-Aksel Puulmann 4af7019011
Ignore sessions without entry/exit pages when breaking down entry/exit pages (#3933)
* Ignore sessions without entry/exit pages when breaking down entry/exit pages

* Update stats controller tests to have more realistic test data (pageview followed by event)
3 years ago
hq1 2fae0146a4
Reapply 3918 (#3940)
* Reapply "Pages shield (#3918)"

This reverts commit 33b5c10654.

* Make the FF check work against the site actor
3 years ago
hq1 9989ce6927
Migration for 3918 (#3939)
* Revert "Pages shield (#3918)"

This reverts commit 53f94a9f82.

* Migration: Shield page rules
3 years ago
hq1 53f94a9f82
Pages shield (#3918)
* Migration: Shield page rules

* Add Ecto schema for Page Rules

* Add Page Rule cache

* Fix typo

* BTW: Use already imported function

* Extend Shields context interface + split existing tests

* Ingestion: filter matching patches + refactor shield actions

* Add LV section for adding Page Rules

* Validate max page path length

* Put Pages Shield behind a feature flag

* Update CHANGELOG

* Update docs link anchor

As per https://github.com/plausible/docs/pull/477

* Update lib/plausible/shields.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible_web/live/shields/page_rules.ex

Co-authored-by: ruslandoga <doga.ruslan@gmail.com>

* Update lib/plausible_web/live/shields/page_rules.ex

Co-authored-by: ruslandoga <doga.ruslan@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
Co-authored-by: ruslandoga <doga.ruslan@gmail.com>
3 years ago
Adrian Gruntkowski ba5b80a8c0
Add label to site imports and populate it (#3914) 3 years ago
Uku Taht 1d017e86a1
Fix escaping of source filters (#3930)
* Fix escaping of source filters

* CHANGELOG

* Fix typo

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
3 years ago
Adrian Gruntkowski 52c226c428
Add `label` column to `site_imports` schema (#3926) 3 years ago
RobertJoonas fb61f0b425
Capitalize Total Conversions in graph tooltip (#3934) 3 years ago
RobertJoonas d6e1e8bebd
Put total conversions on the graph + goal-filtered CSV export improvements (#3929)
* Add validation for the events metric in main_graph

* Test the already existing events metric support in main-graph

* Put total conversions on the graph

* extract main_graph_csv function (refactor only)

* add total_conversions and conversion_rate to goal-filtered visitors.csv

* update changelog
3 years ago
Uku Taht 561dcd821e
Mask pathanme in filter menu event (#3932) 3 years ago
Uku Taht fd879eeb16
Store referrers from android apps (#3715)
* Store referrers from android apps

* Add test for unknown referrer protocol

* Store android referrer protocol
3 years ago
Uku Taht 8992c8ee07
Add tracking to filter button (#3928) 3 years ago
RobertJoonas c32779a3e5
Timeseries for conversion rate (#3919)
* add conversion rate to Stats API timeseries

* make sure CR can be queried as the only metric

* add a test asserting zeros are returned

* add tests for filtering by other properties at the same time

* Remove unnecessary validation of params

1. It doesn't make to validate `interval` (and its granularity) in all
   endpoints. It's only relevant for the main graph.

2. The plug (renamed to `date_validation_plug`) already makes sure that
   the dates are validated. No need to call the same function again in
   Top Stats and Funnel endpoints.

* add metric validation to main graph

* Add tests for main graph API

* put conversion rate on the graph

* update changelog

* Add revenue metrics into metrics.ex

* make fn private

* avoid setting graph metric to visitors in goal-filtered view
3 years ago
Adrian Gruntkowski d6e81670e4
Unify UA and GA4 import flow into one (#3888)
* Unify GA4 and UA import flow into one

* Clean up property and view data retrieval via Google HTTP APIs

* Turn `Map.get` into `Map.fetch!` in API response processing code

* Bump list account summaries page size limit to max of 200

* Show only views in legacy flow and fix legacy redirect after import start

* Move google analytics import actions tests to a separate module

* Extend Google Analytics controller tests

* DRY up `property?` predicate (h/t @RobertJoonas)
3 years ago
ruslandoga 5f9465614b
Include domain and dates in zip archive filename (#3921)
* include domain and dates in zip archive filename

* adapt to comments
3 years ago
Karl-Aksel Puulmann 32ab138301
Fix issue with name clash (#3925)
Unexpectedly, table.name caused a name clash after CR refactor, so using a unique name
for the output column

Sentry issue: https://sentry.plausible.io/organizations/sentry/issues/5612
3 years ago
Karl-Aksel Puulmann c219652dae
Re-apply Move conversion_rate logic from elixir to clickhouse (#3924)
* Revert "Revert "Move conversion_rate logic from elixir to clickhouse (#3887)"…"

This reverts commit 253fb5d67d.

* Fix issue with missing columns

The issue came from refactoring event:goal UNION ALL logic and trying to move
name select from first to last. If any other tables were joined, the incorrect
item would be used as an array index, causing this issue.

Added a relevant test.
3 years ago
Karl-Aksel Puulmann 253fb5d67d
Revert "Move conversion_rate logic from elixir to clickhouse (#3887)" (#3923)
This reverts commit 1909743b90.
3 years ago
Karl-Aksel Puulmann 1909743b90
Move conversion_rate logic from elixir to clickhouse (#3887)
* Separate out query building from pagination/execution logic.

* Refactor pageview_goals breakdown query, removing index column from results

* Remove zip_columns logic

* Use common pagination util

* Do everything in a single query for breakdowns for goals

* Order in DB

* Make sure column order is identical

* Calculate CR within the goal breakdown query

* Calculate CR for property breakdowns

* WIP: Calculate group CR

* CR with order_by

* Compatibility fix

* Import Ecto.Query and cleanup

* handle total_visitors the same way as add_percentage

* Handle conversion_rate in aggregate.ex

* Solve rebase fail

* Simplify maybe_add_group_conversion_rate

* Add conversion_rate defaults to 0 test

* Add test for conversion_rate should not be calculated with imported data (failing here and on master)

* Dont include imported data when breakdown by prop or goal

* Remove revenue_nils
3 years ago
ruslandoga 64f35bf8f6
make robots.txt restrictive by default (#3905)
* make robots.txt restrictive by default

* add note about cloud robots.txt
3 years ago
ruslandoga 279e89c693
CSV imports (no UI) (#3895)
* encode/decode date range in filenames

* Update lib/plausible/imported/csv_importer.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible/imported/csv_importer.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* drop unused functions

* send failure email if there is no data to export

* use PlausibleWeb.Email.mailer_email_from()

* ensure we get dates from minmax date query

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
ruslandoga 4242b52be4
Allow importing extra config (#3906)
* allow importing extra config

* changelog

* fix typo

* add test
3 years ago
ruslandoga dfcc8d794a
Add space after logo on the index page (#3825) 3 years ago
Karl-Aksel Puulmann 02d2256483
Set exit_page only on pageviews (#3870)
* Set exit_page only on pageviews

* Update tests

* Update entry_page on first pageview

* Update CHANGELOG.md
3 years ago
ruslandoga 07b714a143
Update Sentry (#3843)
* update Sentry

* Sentry.HTTPClient.child_spec is now optional

* Sentry.EventFilter is deprecated

* update sentry to 10.2.0

* fix dialyzer warnings
3 years ago
ruslandoga c08aa29ad8
Remove DISABLE_AUTH deprecation warning (#3904)
* remove DISABLE_AUTH deprecation warning

* changelog
3 years ago
ruslandoga 0a0c7ce165
Remove init-admin.sh (#3903)
* remove init-admin.sh

* changelog
3 years ago
ruslandoga 8c0e94f72c
Fix manual embed width (#3910)
* fix manual embed width

* changelog
3 years ago
ruslandoga 4ae00540dc
Split Elixir CI jobs (#3898)
* split elixir tests

* more understandable matrix

* add unused deps in mix.lock check

* get all deps for static checks
3 years ago
ruslandoga 5e74b1cf74
CSV exports (UI) (#3875)
* ui

* fix redirect link

* improve make minio

* use implicit button form for csv export

* add exports_bucket helper

* read S3_EXPORTS_BUCKET

* supply s3_bucket in export_csv job args

* make plausible_minio use unprotected port

* move s3_csv_export queue to base queues

* Update lib/plausible_web/controllers/site_controller.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Karl-Aksel Puulmann d3586a81e3
UX improvement: don't autofocus on Custom properties on dashboard (#3872)
* UX improvement: don't autofocus on Custom properties on dashboard

Currently when you visit the dashboard with a certain configuration,
it will automatically scroll to the bottom, which is annoying.

To reproduce:
1. localStorage.clear() in console
2. Refresh, open properties at the very bottom
3. Refresh again

Expected behavior:
1. Dashboard shows

Actual behavior:
1. Dashboard scrolls to the bottom

* Changelog entry
3 years ago
hq1 59afa20955
Reapply #3878 + bugfix hit rate tracking (#3891)
* Reapply "Replace caching engine (#3878)" (#3883)

This reverts commit c5881cdc6d.

* Ensure hit rate is tracked on `get_or_store`

* Remove :wx and :observer

* Remove unused deps

* Use `:set` table type
3 years ago
RobertJoonas e8f3946dde
Fix division by zero in imported queries (#3890)
* prevent division by 0 in merge_imported queries

* Revert "fix bounce_rate change bug (#3886)"

This reverts commit 6eef32a8ff.

After 02aa0b2, we can keep on assuming that bounce rate is always numeric.
3 years ago
Adrian Gruntkowski 4d7d88cfec
Implement basics of GA4 import (#3851)
* Implement LV date input using flatpickr

* Implement basics of GA4 import (very dirty WIP)

* Split Google HTTP API into UA and GA4 specific parts

* Add a quick way to record GA4 API responses

* Add first GA4 import fixtures with GA4 Data API responses

* Extract GA4 and UA specific logic form Google API

* Extract UA and GA4 specific actions to distinct controllers

* Add integration test for GA4 importer

* Update GA4 fixtures

* Test GA4 API

* Add debug logging and fix paginating through API results in in GA4 import

* Revert "Implement LV date input using flatpickr"

This reverts commit c696f8ee39d5702f27015c09a4f079ca124cc7bb.

* Fix note
3 years ago
ruslandoga f2350b5165
Add `/tmp/` to .gitignore and simplify s3 cleanups in tests (#3889) 3 years ago
ruslandoga 5a3072ca21
CSV exports (no UI) (#3836)
* csv exports

* use ex_unit's tmp_dir
3 years ago
RobertJoonas 6eef32a8ff
fix bounce_rate change bug (#3886) 3 years ago
RobertJoonas 7641c66a2b
Stats api time on page (#3858)
* add metric validation + support in aggregate

* add a test ensuring comparison works

* disallow time_on_page with a goal filter

* Return time_on_page as `nil` from aggregate API

In case time_on_page cannot be calculated, we'll return it as `nil` from
the Stats API.

This is to make the behaviour consistent between breakdown and aggregate
endpoints. As for the UI, we'll still continue to report time_on_page as
0 - not changing any UI behaviour as discussed with Marko.

* add tests for time_on_page in event:page breakdown

* update changelog

* invalidate time_on_page with event:name filter

* add the ability to only query time_on_page in page breakdown

We'll need the visitors metric to get the list of pages to calculate the
time_on_page for.
3 years ago
Cenk Kücük dea9287d74
Revert "Temporarily disable tzdata auto-updating (#3873)" (#3885)
This reverts commit 11368e5143.
3 years ago
hq1 c5881cdc6d
Revert "Replace caching engine (#3878)" (#3883)
This reverts commit 437a3350ff.
3 years ago
Marko Saric ca7edd47cb
removing target="_blank" from blog link (#3882) 3 years ago
hq1 437a3350ff
Replace caching engine (#3878)
* Dependencies: swap Cachex for ConCache

* Implement Cache adapter wrapping ConCache

* Implement cache stats tracker, for metrics

* Use Cache.Adapter in Plausible.Cache

Marking the test as not slow anymore

* Use Cache Adapter when tracking sessions

* Use Cache Adapter for UA parsing

* Rename child identifiers - cachex is obsolete now

* Test stats tracking

* Update grafana metrics

* Put all caches under common child specification

* Try less

* Shorten the function delegation path
3 years ago
hq1 bb8a272a36
Fix margins in help guides section (#3881) 3 years ago
hq1 b68e3aa6a2
changes to the in-app footer (clean) (#3880)
* changes to the in-app footer

changes to the in-app footer to focus on product usage links rather than marketing pages

* Format

---------

Co-authored-by: Marko Saric <34340819+metmarkosaric@users.noreply.github.com>
3 years ago
ruslandoga 83a46fb47e
Dockerfile and co updates (#3811)
* wip

* more env, setup user before packages to avoid them picking our uid

* make entrypoint.sh executable by all again

* apparently it's a best practice to have executables be owned by root

* make entrypoint executable in COPY

* stop writing to /app, write to /tmp if needed

* fewer changes

* system user

* keep same style for multiline commands

* fewer changes

* add changelog entry

* fix group assignment for plausible user

* use gid=999

* no home

* no home

* add gecos

* add plausible user to nogroup instead of creating a custom one

* eh

* fewer changes

* fewer changes

* fewer changes

* use PERSISTENT_CACHE_DIR instead of STORAGE_DIR

* ignore more

* cleanup

* remove hex timeout env var

* use ERL_FLAGS=+JMsingle true in public builds

* fallback to /tmp and nest under /tzdata_data/ for tzdata

---------

Co-authored-by: Cenk Kücük <cenk@plausible.io>
3 years ago
Karl-Aksel Puulmann 11368e5143
Temporarily disable tzdata auto-updating (#3873)
This can cause issues as 1 day after a deploy the tzdata list is updated
from 2021 to 2024. Selecting a timezone thats not supported in 2021 can
then break dashboards temporarily following a deploy.

This can be reverted after https://github.com/plausible/analytics/pull/3811
3 years ago
Karl-Aksel Puulmann a9d3c03782
Validate the same metric isnt queried multiple times in external stats API (#3871)
* Validate the same metric isnt queried multiple times in external stats API

Issue: https://3.basecamp.com/5308029/buckets/35611491/card_tables/cards/7161347855

* Changelog entry

* Make credo happy
3 years ago
Karl-Aksel Puulmann 26d41ddbb9
Mutation to populate event session columns (#3844)
* WIP mutation to populate event session columns

* Remove duplication

* report errors, allow_nondeterministic_updates

* use right columns

* Update existing columns instead of session_* ones

* Make dialyzer happy

* Fix issue with passing pre-existing params in

* Logger -> IO.puts

* Use IngestRepo.config for connection settings

* Make dictionary options configurable

* Move allow_nondeterministic_mutations to within the migration

* Solve credo warning about too deep nesting

* Missed logger call

* Pattern matching in function head
3 years ago
Karl-Aksel Puulmann 0cdba7d407
Fix broken tests (#3867) 3 years ago
Karl-Aksel Puulmann c6d98397a8
Move `add_percentage` logic into clickhouse (#3854)
* Remove `add_percentage`, calculate percentages in clickhouse queries

This simplifies querying logic and avoids doing extra queries and avoids
race conditions.

* Remove special none handling from breakdowns, handling percentages correctly

* Add (failing) test showing expected add_percentage behavior for user making multiple sessions

* Update add_percentage behavior to use separate subqueries
3 years ago
Karl-Aksel Puulmann c60a2faee4
Write event table session columns (#3865)
* Write event table session columns

* Update testing factory rig
3 years ago
Karl-Aksel Puulmann 70d21f1b69
Improve compression of event table session columns (#3864) 3 years ago
Karl-Aksel Puulmann 8d977e0f76
Tests: session properties without the prefixes (#3863)
* Undo event session attributes renaming

* Rename session_ attributes in tests
3 years ago
Cenk Kücük fecd665c19
Push private images to GHCR (#3859) 3 years ago
Karl-Aksel Puulmann d5048fd6b4
Stop writing session properties into events table (#3800)
* Refactor: Explicitly add field names to INSERT

This avoids issues when code schema is out of sync with real schema

* Dont write session parameters to events

These would only be stored on first event anyways. Work remains to be done
on tests which have their own helper

* Remove writes to country_code in a test

* Remove old columns from being accessible in elixir code

* Update most tests to use new way of adding session props to events

* Update testing harness

* Update stats controller test

* Update for shield rules

* update breakdown tests

* Fix typing of state for dialyzer

* Drop support for old session attributes code

* Update remaining tests

* cond -> if
3 years ago
hq1 5eb9d724e5
Plugins API: mark data_domain as nullable in capabilities schema (#3840)
* Plugins API: mark data_domain as nullable in capabilities schema

* fixup
3 years ago
Karl-Aksel Puulmann ea38b45685
Ecto migration to move sessions_v2 to VersionedCollapsingMergeTree (#3809)
This migration will noop in staging/production as it already has been run. It also leaves
behind a backup table that initially takes no extra space but will need to be cleaned up
manually
3 years ago
Adrian Gruntkowski 39aa81a16f
Implement UI for multiple imports (#3727)
* Create a stub of site settings section for imports and exports

* Use legacy site import indication to determine UA import handling

* Add provisional logos for upcoming import sources

* Stub basics of import page

* Add very rudimentary support for multiple UA imports

* Implement imports list as live view

* Add support for opening LV modal from backend and closing from frontend

* Introduce notion of themes to `button` and `button_link` components

* Add confirmation modal on deleting import

* Swap GA4 logo

* Implement disabled state support for `button_link` component

* Disable export and non-implemented import sources

* Use native starts start date for upper boundary of import time range

* Ensure integrations view uses legacy UA import flow

* Remove unnecessary preload in SiteController

* Remove unnecessary exception for legacy imports

* Move API controller stats tests under PlausibleWeb

* Test listing imports

* Add test for explicit listener setup

* Add tests for legacy flag state in UA importer

* Add test for purging legacy import data

* Add tests for `Sites.native_stats_start_date`

* Test forgetting imports

* Add `Stats.Clickhouse.imported_pageview_counts/1` and fix test flakiness

* Show page view counts on imports list

* Add tests for static imports and exports view

* Adjust button look slightly

* Use `case` instead of `cond`

* Make feature flag customisable per site

* Fix buttons and empty state styling

* Add another import to seeds

* Use JS confirm dialog instead of modal for deletion confirmations

* Revert "Add support for opening LV modal from backend and closing from frontend"

This reverts commit 260e6c753032b451542e24be9edc2118790b5a00.

* Default `legacy` to false when inserting new import jobs

* Drop `method` attribute from `button_link` and `unstyled_link` components
3 years ago
Adrian Gruntkowski fdbe4cc0a4
Add `legacy` flag column to `site_imports` schema (#3842) 3 years ago
Marko Saric 0fe5682f91
Change the CTA to upgrade now rather than contact (#3837) 3 years ago
hq1 316148e788
Fix settings options mobile (#3838)
* Fix settings options on mobile view

* Fix settings options selected item on mobile

* Format
3 years ago
ruslandoga f3423aefec
Add csv importer (#3795)
* add csv importer

* make table validation explicit

* update some docs

* improve docs

* add minio container to ci

* more tests

* eh

* continue

* add passing test

* add failing test

* add config test

* add minio to Makefile

* testcontainers

* remove extra whitespace

* explain the implementation a bit

* account for async deletes in tests

* bounces is UInt32

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
ruslandoga 31cf3e54f8
Add Bamboo.Mua (#3654) 3 years ago
hq1 f1b6a672d4
Fix `added_by` not saving on adding country rules (#3835)
* Fix `added_by` not saving on adding country rules

* Format

* Remove dupe attr
3 years ago
hq1 518cdb3307
Shield: Country Rules (#3828)
* Migration: add country rules

* Add CountryRule schema

* Implement CountryRule cache

* Add country rules context interface

* Start country rules cache

* Lookup country rules on ingestion

* Remove :shields feature flag from test helpers

* Add nested sidebar menu for Shields

* Fix typo

* IP Rules: hide description on mobile view

* Prepare SiteController to handle multiple shield types

* Seed some country shield

* Implement LV for country rules

* Remove "YOU" indicator from country rules

* Fix small build

* Format

* Update typespecs

* Make docs link point at /countries

* Fix flash on top of modal for Safari

* Build the rule struct with site_id provided up-front

* Clarify why we're messaging the ComboBox component

* Re-open combobox suggestions after pressing Escape

* Update changelog

* Fix font size in country table cells

* Pass `added_by` via rule add options

* Display site's timezone timestamps in rule tooltips

* Display formatted timestamps in site's timezone

And simplify+test Timezone module; an input timestamp converted
to UTC can never be ambiguous.

* Remove no-op atom

* Display the maximum number of rules when reached

* Improve readability of remove button tests

* Credo

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
hq1 b7b5dcf4c9
Migration: add country rules (#3834)
* Migration: add country rules

* Make `country_code` not nullable
3 years ago
Cenk Kücük 8a8df685a7
Update country code for anonymising VPN services (#3766)
* Update country code for anonymising VPN services

* Update to latest
3 years ago
RobertJoonas dd428430f5
Query imported data for views_per_visit (#3830)
* query imported data for views_per_visit

* changelog update
3 years ago
Cenk Kücük cc867396b2
Update elixir.yml (#3831)
Only test against the last 2 major versions
3 years ago
ruslandoga bea8ceeffb
use newer postgres versions in the ci (#3821) 3 years ago
hq1 c822f8de1e
Hide user name overlay on mobile (#3829) 3 years ago
Karl-Aksel Puulmann c6e06359c6
Avoid changing Custom properties tab automatically every time filter changes (#3777)
Follow-up to https://github.com/plausible/analytics/pull/3719

The previous behavior was predicated on:
- Allowing a single custom property filter
- Allowing breaking down only by the chosen custom property filter

Now these restrictions are removed the previous auto-switching just becomes annoying
3 years ago
Marko Saric 19ce62db61
Update README.md 3 years ago
Uku Taht a1fedb751b Update license hash 3 years ago
Cenk Kücük 0f0f604d2e
Branding stuff (#3816)
* Replace footer text

* add COPYING.txt file

* Add new logos

* Use new logos in all layouts

* New logos

* Check license key on startup

* Bypass license check when Mix.env == :dev

* Use new logos with smaller wordmarks

* Add generic logo_path/1 function

* Use new favicons everywhere

* Bypass license check in test env

* Use sha256 for license key hash

* Mix.env -> config_env()

* Use Mix.evn at compile time rather than runtime

* Mix format

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
3 years ago
Cenk Kücük bdaf80c05d
Publish community edition under plausible/community-edition on GHCR and dockerhub (#3815) 3 years ago
Karl-Aksel Puulmann acd4f33c8e
Migration to improve compression options of sessions_v2 and events_v2 tables (#3803)
Part of https://3.basecamp.com/5308029/buckets/35611491/messages/7061875211
Based on work in https://3.basecamp.com/5308029/buckets/35611491/messages/6949640880#__recording_7015576045
3 years ago
hq1 bca617d0bf
`String.to_atom/1` is safe to do at compile-time (#3813)
otherwise random compile-time errors may occur
3 years ago
RobertJoonas 52f584efa9
Group `os_version` by `os` (#3806)
* improve test

* add os to os_version breakdown

...and add operating_system_versions.csv to the CSV export

* fix conversion rate for os_version breakdown

* update changelog

* fix existing CSV tests

* use case instead of cond
3 years ago
Karl-Aksel Puulmann 8ba2b934b3
Add data migration for moving sessions_v2 table to VersionedCollapsingMergeTree (#3802)
* Add data migration for moving to VersionedCollapsingMergeTree

This has been tested locally and partially on staging. Still requires a bit of work to verify.

Verification query:

```
SELECT main._partition_id, tmp.count, main.count
FROM (
SELECT _partition_id, count() AS count
FROM sessions_v2_tmp_versioned
GROUP BY _partition_id
) AS tmp
FULL OUTER JOIN (
SELECT _partition_id, count() AS count
FROM sessions_v2
GROUP BY _partition_id
) AS main
ON (tmp._partition_id == main._partition_id)
ORDER BY main._partition_id
```

* Add an early exit to migration

* cluster? extract common code
3 years ago
RobertJoonas d74b1d5e60
Reapply conversion rate into Stats API + bugfixes (#3805)
* Revert "Revert api conversion rate (#3789)"

This reverts commit 8e8790dd30.

* fix browser_version CR breakdown bug

* changelog bugfix

* inspect data structures before sending to sentry
3 years ago
hq1 6035618213
Add `GET /capabilities` to Plugins API (#3808)
* Add `GET /capabilities` to Plugins API

It aims to:

 - help the client verify the data-domain the token is associated with
 - list all the features available for the site's owner
   (and therefore determine availability of the subset of those for the current
   Plugins API caller)

The endpoint does not require authentication, in the sense that it'll
always respond with 200 OK. However when the token is provided,
a verification lookup is made.

* Remove IO.inspect() call

* Credo

* Aesthetics

* s/send_resp/send_error/

* Call preload just once
3 years ago
hq1 23b9032148
Allow site deletion with IP Rules stored (#3804)
* Allow site deletion with IP Rules stored

* drop_if_exists
3 years ago
Karl-Aksel Puulmann 5cdca6f408
Remove code that updates session browser/geo/OS attributes (#3796)
This will makes it impossible to store session attributes on events.

Looking at production data also revealed no cases where these updates
are effective.
3 years ago
hq1 eceac8afd5
Allow inviting users who are members already (#3797)
* Allow e-mail exclusion in team members quota

* Exclude invitee from quota on invitation create

* Enable invitation submission but report errors on quota violation

* Use a single interface for team members quota

* Check the `Keyword.validate/2` result

* Update test/plausible_web/controllers/site/membership_controller_test.exs

Co-authored-by: Uku Taht <Uku.taht@gmail.com>

---------

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
3 years ago
Adrian Gruntkowski e96a52c588
Revert "push docker images to ghcr.io (#3787)" (#3790)
This reverts commit c430274d7e.
3 years ago
RobertJoonas 8e8790dd30
Revert api conversion rate (#3789)
* Revert "Unify percentage change for CR and bounce_rate (#3781)"

This reverts commit a6b1a6ebc7.

* Revert "Bring Stats API up to speed: Add `conversion_rate` to Aggregate and Breakdown (#3739)"

This reverts commit 672d682e95.
3 years ago
Cenk Kücük c430274d7e
push docker images to ghcr.io (#3787) 3 years ago
RobertJoonas a6b1a6ebc7
Unify percentage change for CR and bounce_rate (#3781)
* Fix conversion rate change calculation

The change in conversion rate should be calculated similar to bounce rate.
For example, an increase of 25% -> 50% should not be a 100% change, but
a 25% change instead.

* Use the same comparison function in Stats API and dashboard API

This commit fixes a bug where the percentage change reported by the Stats
API is different from the one returned by the internal dashboard API.

* changelog update
3 years ago
hq1 926de4dd10
Experimental session count (#3786)
* WIP

* Allow `experimetnal_session_count` request serialization

* Extend `Plausible.Stats.Query` with `experimental_session_count` flag

* Add `FunWithFlags` actor implementation for `Site`

* Change the way sessions are retrieved

* Remove redundant test

* Format

* Update the test

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
3 years ago
RobertJoonas 672d682e95
Bring Stats API up to speed: Add `conversion_rate` to Aggregate and Breakdown (#3739)
* disable event metric with include_imported in every case

* add missing test for metric validation

* refactor metric validation functions

* implement conversion_rate metric validation

* move calculate_cr function into Stats.Util

* Refactor: Move aggregate CR logic into Stats.aggregate

* define atoms to exist

* Ensure that CR does not depend on visitors being queried

If 'visitors' are already queried, we'll use that value. Otherwise we'll
need to make another query to fetch it.

* confirm Stats API aggregate supports CR (tests only)

* small refactor

This is the only 'event_property' left after pattern matching on all
others in the function clauses defined above.

* Make it possible to optionally query conversion_rate

...in breakdown queries (excluding goal and custom prop breakdown)

* A little refactor asking for revenue metrics

1. The `@revenue_metrics` module attribute is an empty list on full build
   anyway
2. We don't need to query for revenue metrics if there are no revenue goals
   returned in the given query (even if revenue goals exist in site.goals)
3. Revenue metrics are already dropped in prop breakdown without a goal
   filter via (get_revenue_tracking_currency/3)

* Make it possible to optionally query conversion_rate (continuation)

... also from a custom prop and goal breakdown

* Frontend adjustments to the Locations report

* Display conversion rate in Regions and Cities (ListReport view)
* Display total conversions, conversions (visitors), and CR in the
  "Details" modals of Countries, Regions, and Cities
* Move the percentage into a separate column in the Countries details table

* confirm Stats API breakdown supports conversion_rate (tests only)

* small refactor: extract maybe_add_time_on_page function

* Make it possible to query cr alone

... (without the visitors metric). Already supported in aggregate, this
commit only implements it for the breakdown API.

* Reuse Stats.Util helper functions from b02db88 for aggregate API

We can follow the same logic as with breakdown for manually adding
`visitors` into the metrics list and taking it out of the response
later on.

That way we don't have to make another query, e.g. in a case where
only pageviews and conversion rate is queried. Also keeps things
consistent.

* changelog update

* fix test after resolving merge conflict

* Use explicit string->atom mapping instead of casting

* alias Util module instead of importing it

* use Enum.empty instead of Enum.any

* improve readability

* rename special_metrics to computed_metrics and explain with a comment

* rename visitors_without_event_filters to total_visitors

* keep a single function for removing unwanted metrics

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Karl-Aksel Puulmann 0065cd3052
Support using matches/contains for most filters (#3721)
* Support using matches/contains for most filters

* Change behavior where we auto-zoom to specific browser/os/source to only do so if filtering on a single value

* No contains filtering on `location`

* Update CHANGELOG.md

* Fix merge conflict
3 years ago
Karl-Aksel Puulmann ac7da6a9d4
Set materialized_views_ignore_errors (#3784)
We had an incident where a stray materialized view caused inserts to fail. This setting
causes stray mv errors to silently fail, which is okay for our current usecases.

Further reading: https://clickhouse.com/docs/en/sql-reference/statements/create/view#materialized-view
3 years ago
ruslandoga 423f72a0ad
safe(r) Oban telemetry (#3743)
* safe telemetry

* use try/catch instead

* add some tests

* cleanup tests

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
Adrian Gruntkowski f8b4d5066a
Add multiple imports per site (#3724)
* Clean up references to no longer active `google_analytics_imports` Oban queue

* Stub CSV importer

* Add SiteImport schema

* Rename `Plausible.Imported` module file to match module name

* Add `import_id` column to `Imported.*` CH schemas

* Implement Importer behavior and manage imports state using new entities

* Implement importer callbacks and maintain site.imported_data for UA

* Keep imports in sync when forgetting all imports

* Scope imported data queries to completed import IDs

* Mark newly imported data with respective import ID

* Clean up Importer implementation a bit

* Test querying legacy and new imported data

* Send Oban notifications on import worker failure too

* Fix checking for forgettable imports and remove redundant function

* Fix UA integration test

* Change site import source to atom enum and add source label

* Add typespecs and reduce repetition in `Plausible.Imported`

* Improve documentation and typespecs

* Add test for purging particular import

* Switch email notification templates depending on import source

* Document running import synchronously

* Fix UA importer args parsing and ensure it's covered by tests

* Clear `site.stats_start_date` on complete import to force recalculation

* Test Oban notifications (h/t @ruslandoga)

* Purge stats on import failure right away to reduce a chance of leaving debris behind

* Fix typos

Co-authored-by: hq1 <hq@mtod.org>

* Fix another typo

* Refactor fetching earliest import and earliest stats start date

* Use `Date.after?` instead of `Timex.after?`

* Cache import data in site virtual fields and limit queried imports to 5

* Ensure always current `stats_start_date` is used

* Work around broken typespec in Timex

* Make `SiteController.forget_imported` action idempotent

* Discard irrecoverably failed import tasks

* Use macros for site import statuses

There's also a fix ensuring only complete imports are considered
where relevant - couldn't isolate it as it was in a common hunk

* Use `import_id` as worker job uniqueness criterion

* Do not load imported stats data in plugins API context

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
Adrian Gruntkowski 6915dd5e78
Add table migrations for supporting multiple imports per site (#3785)
* Add migration deleting jobs from legacy analytics import queue

* Add `site_imports` table

* Add `import_id` column to imported_* CH tables

* Extend one of `site_imports` indexes
3 years ago
Marko Saric 05adf5d83c
copy changes to the CTA on the top of a new dashboard (#3783)
* Update stats.html.eex

* Update stats.html.eex
3 years ago
Karl-Aksel Puulmann f5e15941de
Add migration for minmax indexes on session timestamps (#3772) 3 years ago
Karl-Aksel Puulmann f3509f2a17
Refactor spike detection top sources query (#3770)
* ORDER BY referrer_source for spikes job

This is more consistent with the rest of the queries

* Refactor top_sources -> top_sources_for_spike

* Remove more dead code

* Remove unused arguments

* Remove unused select arguments

* Add a test to top_sources_for_spike
3 years ago
Karl-Aksel Puulmann d1fe184cb7
Remove ORDER BY min(session.start) (#3771)
This adds a needless performance overhead. Note that all queries already
sort by the _name_ of the breakdown value besides the session start.
3 years ago
hq1 494ffbd622
Shields behind a FF (#3780) 3 years ago
hq1 99fe03701e
IP Block List (#3761)
* Add Ecto.Network dependency

* Migration: Add ip block list table

* If Cachex errors out, mark the cache as not ready

* Add IPRule schema

* Seed IPRules

* Add Shields context module

* Implement IPRuleCache

* Start IPRuleCache

* Drop blocklisted IPs on ingestion

* Cosmetic rename

* Add settings sidebar item

* Consider IPRuleCache readiness on health checks

* Fix typo

* Implement IP blocklist live view

* Update moduledocs

* Extend contextual module tests

* Convert IPRules LiveView into LiveComponent

* Keep live flashes on the tabs view

* Update changelog

* Format

* Credo

* Remove garbage

* Update drop reason typespecs

* Update typespecs for cache keys

* Keep track of who added a rule and when

* Test if adding via LV prefills the updated_by tooltip

* Update ecto_network dependency

* s/updated_by/added_by

* s/drop_blocklist_ip/drop_shield_rule_ip

* Add docs link

* s/Updated/Added
3 years ago
hq1 bc467996ab
Shields IP blocklist: migration (#3779) 3 years ago
hq1 6a2d7fc0f5
Merge `Plugins.API.Router` into main one (#3767)
* Merge `Plugins.API.Router` into main one

In order to get grafana metrics reported
See: https://github.com/akoutmos/prom_ex/issues/224

* Format
3 years ago
hq1 f5129f1b0d
Turn Revenue Goals into Custom Events if the plan doesn't support them (#3768)
* Turn Revenue Goals into Custom Events if the plan runs out

* Tag test with full_build
3 years ago
Karl-Aksel Puulmann 499be9f0b3
Add more custom properties to be tracked (#3778)
We just released support for multiple custom property filters, but our demo/live setup
does not have many useful custom properties. This PR adds a few more to allow
better slicing-and-dicing.
3 years ago
Karl-Aksel Puulmann 1cb7982cd9
Filtering by multiple custom properties (#3719)
* WIP: PropFilterRow

* Get multi-behavior working

* Render multiple prop filters in one

* Modal reads from query string correctly

* Backend support for multiple custom property filters

* Add backend tests for multiple custom property filters

* Disable already selected options in property keys

We can't allow choosing the same property multiple times without changing the request
params, which we decided against

* Allow choosing any property under Behaviors > Custom props even if custom prop filter applied

This was a limitation (I believe) introduced by using ARRAY JOINs to query custom properties

* CHANGELOG.md

* Solve credo warning about too deep nesting

* Update assets/js/dashboard/stats/modals/prop-filter-modal.js

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

* Refactor internal function for clarity

* Add another step -> Add another

* Solve 500 error

* Separate boxes per property filter

* Retain other filters in props table

* removeFilter behavior for props

* matches_member support for custom props

* filter_suggestions for prop keys should account for prop filter

* find over filter

* refactor appliedFilters

* FILTER_TYPES => FILTER_OPERATIONS

* Make add another link not wrap the whole page

* Unique keys

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
dependabot[bot] c8b25347ed
Bump buildjet/cache from 3 to 4 (#3753)
Bumps [buildjet/cache](https://github.com/buildjet/cache) from 3 to 4.
- [Changelog](https://github.com/BuildJet/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/buildjet/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: buildjet/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cenk Kücük <cenk@plausible.io>
3 years ago
hq1 9f2f894421
Fix flashing on reapplying theme (#3760)
* Fix flashing on reapplying theme

This seem to be making a difference on staging

* Add `blocking="rendering"` attribute
3 years ago
hq1 6ce4d19931
Fix flaky test: wait till ets is emptied (#3755)
* Fix flaky test: wait till ets is emptied

* fixup
3 years ago
hq1 9ad5ecebc2
Improve print dashboard css (#3754)
* Improve print dashboard CSS

* More improvements
3 years ago
Marko Saric 03e1224693
improvements to the email copy (#3756)
* Update yearly_expiration_notification.html.eex

* Update yearly_renewal_notification.html.eex
3 years ago
RobertJoonas 51f0e406a0
improve stats api upgrade required error message (#3747) 3 years ago
Karl-Aksel Puulmann 625b2a2946
Use parallel_hash for joins in clickhouse (#3741) 3 years ago
Adrian Gruntkowski 3738cd9578
Parse referrer filter value when passed separately (#3742)
* Parse referrer filter value when passed separately

* Remove unnecessary test setup
3 years ago
hq1 e28793a45a
Make `hello@plausible.io` clickable (#3746)
* Make `hello@plausible.io` clickable

* Format

* ...
3 years ago
hq1 9fb4ea0c3e
Bugfix: available features for expired trials, no subscriptions (#3740)
* Reorganize how subscriptions/trials are evaluated

* Bugfix: expired trial+no subscriptions should not have access to extra features

* Make self-hosted users always on trial

* Seed secondary user with password

* Format

* Fix docs

* Fix small_test run

* Run the test only on full_build

* More tweaks to small builds

* Allow [Goals] for expired trials with no subscription
3 years ago
dependabot[bot] 85c9771d11
Bump @playwright/test from 1.40.1 to 1.41.1 in /tracker (#3744)
Bumps [@playwright/test](https://github.com/microsoft/playwright) from 1.40.1 to 1.41.1.
- [Release notes](https://github.com/microsoft/playwright/releases)
- [Commits](https://github.com/microsoft/playwright/compare/v1.40.1...v1.41.1)

---
updated-dependencies:
- dependency-name: "@playwright/test"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
dependabot[bot] 63c4c1d07b
Bump eslint from 8.55.0 to 8.56.0 in /tracker (#3640)
Bumps [eslint](https://github.com/eslint/eslint) from 8.55.0 to 8.56.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/compare/v8.55.0...v8.56.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
dependabot[bot] c86959ef9e
Bump ex_money from 5.15.2 to 5.15.3 (#3696)
Bumps [ex_money](https://github.com/kipcole9/money) from 5.15.2 to 5.15.3.
- [Release notes](https://github.com/kipcole9/money/releases)
- [Changelog](https://github.com/kipcole9/money/blob/main/CHANGELOG.md)
- [Commits](https://github.com/kipcole9/money/compare/v5.15.2...v5.15.3)

---
updated-dependencies:
- dependency-name: ex_money
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: hq1 <hq@mtod.org>
3 years ago
hq1 931161f693
Generalize Cache/Warmers - extract only specifics to `Site.Cache` (#3716)
* Simplify caches config

The intervals were proven to never change really,
and previous shape made them not very discoverable either.
We'll look into a more declarative setup.

* Aim for more declarative supervision tree setup

* Generalize `Plausible.Cache`

This makes the `Site.Cache` module implement the specifics,
leaving the common bits reusable for upcoming cache processes.

* Generalize `Plausible.Cache` warmers

* Fix typos
3 years ago
ruslandoga d5818a63f7
update Ch (#3733) 3 years ago
dependabot[bot] 3c9062a1dd
Bump actions/cache from 3 to 4 (#3734)
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cenk Kücük <cenk@plausible.io>
3 years ago
hq1 18ddf1e7a0
Re enable accept traffic until notifications (#3737)
* Revert "Remove 0-sized queue"

This reverts commit 6d229b3f70.

* Revert "Temporarily remove notification cron worker (#3732)"

This reverts commit 447e3929cb.

* Revert "Temporarily disable accept_traffic_until actions (#3730)"

This reverts commit 58afe3376f.

* Send accept_traffic_until notifications at 8 UTC rather than 5

* Add option for dry-run

* Expose dry_run in a dedicated function
3 years ago
hq1 ee6d16d44f
Shift accepted_traffic_until for possibly delayed notifications (#3738) 3 years ago
hq1 8f6ec252e9
Backfill accepted_traffic_until for active subscriptions (#3736)
* Backfill accepted_traffic_until for active subscriptions

* Narrow the results
3 years ago
RobertJoonas 04b9067591
Add `event:goal` property to Stats API (#3718)
* add tests for filtering by goal in timeseries and aggregate

* refactor filter parsing

* stop returning custom props in event:goal breakdown

* test breaking down wildcard pageview goals

* extract filter utils

* parse more goal filter options

* add passing tests for new filter types

* do not allow querying session metrics with a goal filter

* remove unused page_match property

* test that non-configured goals are not returned in breakdown

* enforce filtered goals configured

* update changelog

* Allow simple filtering by revenue goals

This does not mean that revenue metrics are supported. If a revenue goal
is filtered by, we treat it like a simple custom event goal in the API.

* use List.wrap
3 years ago
Adam Rutkowski 6d229b3f70 Remove 0-sized queue 3 years ago
Adam Rutkowski 00ef386356 Revert "Bump actions/cache from 3 to 4 (#3707)"
This reverts commit 100e89370f.
3 years ago
hq1 447e3929cb
Temporarily remove notification cron worker (#3732)
* Temporarily remove notification cron worker

* Format
3 years ago
hq1 712f60f37f
Update `accept_traffic_until` on payment succeeded notification (#3731)
* Update `accept_traffic_until` on payment succeeded notification

* full build only
3 years ago
hq1 58afe3376f
Temporarily disable accept_traffic_until actions (#3730)
* Temporarily disable accept_traffic_until actions

* Remove commented code
3 years ago
dependabot[bot] 100e89370f
Bump actions/cache from 3 to 4 (#3707)
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cenk Kücük <cenk@plausible.io>
3 years ago
hq1 0fa7f3c1e1
Make subsequent query boundaries idempotent (#3723)
* Make subsequent query boundaries idempotent

This should prevent time-sensitive race conditions
on subsequent `Plausible.Stats.*` calls for the same query.
Not sure if there are more places where now() is
dynamic relative to query life cycle, but it's a start.

* Format

* Update lib/plausible/stats/query.ex

Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>

* Match on `Query.now` in a test

---------

Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>
3 years ago
Adrian Gruntkowski 822483c37c
Extract Universal Analytics import logic (#3700)
* Move imported tables schemas to separate modules outside Google ns

* Move buffer for imports to Imported ns

* fix schema newlines

* Extract UA import processing and persistence

* Decouple analytics worker implementation from UA

* Rename env variable for import buffer size

* Preserve old import queue until release
3 years ago
Karl-Aksel Puulmann 11c5d3b251
Remove ARRAY JOINs (#3708)
* Use array indexing instead of left joins

This has multiple benefits:
- Allows us to more easily add support for multiple custom property
  filters
- Performance: Increased performance of some queries by ~20%

Work left to do:
- Check tests coverage
- Create macros for repeated fragments
- Quote and refer to right tables in fragments/macros

* Use macros to clean up code

* Use existing atoms to avoid memory leaks
3 years ago
Karl-Aksel Puulmann e2eb861799
Github Actions: Run tests on all PRs (#3709)
Chained PRs (like https://github.com/plausible/analytics/pull/3708)
don't currently run tests. This PR fixes that.
3 years ago
RobertJoonas 041b0ab779
trace aggregate and timeseries queries too (#3712) 3 years ago
hq1 36a8b36382
CombBox bugfix: stale spinner on creatable options selection (#3713) 3 years ago
RobertJoonas 6ca4eaf5e2
Improve query tracing (#3704)
* Enable breakdown property tracing for event:goal

This is currently overwritten by event:name. The queries for event:name
will still be tracked if queried specifically.

* Add filters tracing to Query.trace/1

* make sure breakdown metrics are sorted and joined

* check opts skip tracing directly without passing it
3 years ago
hq1 8965ac40a7
Decorate site settings sidebar with heroicons (#3711)
* Decorate site settings sidebar with heroicons

* Capitalize "Danger Zone"

* Capitalize "Danger Zone"
3 years ago
Karl-Aksel Puulmann 1b95433d6e
Support `contains`/`matches` for custom properties (#3687)
* Allow `matches` operator to work in BE for custom props

Note: No FE support yet, needs further testing

* feat: allow choosing `contains` for property filters in the UI

* no autocomplete on prop values if `contains` for consistency

* CHANGELOG.md

* Fix: Handle (none) property in property breakdowns when using matching

When matching we should always exclude (none)
3 years ago
hq1 0a124e69e7
Actually enforce proper WordPress spelling (#3710) 3 years ago
Uku Taht 730c8edc4a
Update ref_inspector database (#3697)
* Update ref_inspector database

* Use random referrer input
3 years ago
Uku Taht 024efd1fc1
Update ua_inspector database (#3698)
* Update ua_inspector source

* Downloan device detector updates
3 years ago
Marko Saric fb80847d7e
capital P in WordPress (#3703) 3 years ago
Arkadiusz Plichta 4ec86abeac
update clickhouse docker image (#3701) 3 years ago
Uku Taht a5e0619053
Escape special regex characters (#3634)
* Escape special regex characters

* Add changelog entry
3 years ago
dependabot[bot] cba5f55ba1
Bump ex_doc from 0.31.0 to 0.31.1 (#3693)
Bumps [ex_doc](https://github.com/elixir-lang/ex_doc) from 0.31.0 to 0.31.1.
- [Release notes](https://github.com/elixir-lang/ex_doc/releases)
- [Changelog](https://github.com/elixir-lang/ex_doc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/elixir-lang/ex_doc/compare/v0.31.0...v0.31.1)

---
updated-dependencies:
- dependency-name: ex_doc
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
hq1 7deffcdeee
Prefer example.com in tests (#3690) 3 years ago
Cenk Kücük 44fca0c95d
Prioritize X-Plausible-IP header over everything else (#3689) 3 years ago
RobertJoonas 0a29679afa
Stop expecting `trial_expiry_date` to exist on upgrade (#3688)
* stop expecting trial_expiry_date to exist on upgrade

* add test
3 years ago
RobertJoonas d3094ffdb7
Prevent upgrades that would leave the customer locked (#3683)
* Remove allowance_required field from grace_period

Since we are now preventing customers from subscribing to a plan that
does not accommodate their pageview usage, there is no need for an extra
check on removing the grace period after a successful upgrade.

This extra check is the reason why the automatic unlocks have recently
failed in several cases.

* refactor outgrown subscription notices

* make a test actually test the described functionality

* Apply greater pageview allowance margin only for trial upgrades

...in order to prevent cancelled or paused subscriptions from subscribing
to plans that would still leave their account locked.

* Mark the entire ChoosePlanTest module full build only

* remove account locking guide

This is irrelevant for self-hosters, and the internal knowledge base is
a better place for this document. Moved it there.

* refactor Keyword get clause

* add a pattern matching assertion in code
3 years ago
RobertJoonas 7536f0285c
Fix weekly interval bug (#3686)
* Fix a bug

This commit fixes a bug where the timeseries weekly interval query for a
month returned 0 visitors for the first week.

This happened because the start of the month defining the first week was
missing the UTC -> site.timezone conversion.

* Update test/plausible_web/controllers/api/stats_controller/main_graph_test.exs

Remove dot from test description

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

---------

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
3 years ago
Adrian Gruntkowski eaa7020230
Latency-friendly LV modal implementation and Goals form refactor (#3649)
* Make modal for goal settings trigger without BE roundtrip

* Turn goal form into a live component and extract modal into a wrapper

* Further extract modal component and handle reset action

* Make ComboBox selection callback more flexible

* Add rudimentary loading state to dialog

* Make form unaware of being put inside a modal

* Make modal a live component and completely reset contents on open server-side

* Try to avoid race condition

* Fix race condition

* Remove unnecessary conditional on socket assigns

* Add typespecs and fix formatting

* Make goals form high latency friendly

* Fix tests to account for goal settings form becoming live component

* Fix goal settings form live component declaration

* Add documentation for modal

* Fix small build test

* Fix typo

Co-authored-by: hq1 <hq@mtod.org>

* Revert no longer necessary test changes from 46f65d9

* Fix and clean up modal styling

* Keep focus on dialog when open and show only spinner on backdrop when loading

* Adjust corners and shadows and implement open/close transitions

* Lock body scroll when modal is open

* Make modal top-aligned again to avoid jumping around on variable content height

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
Adrian Gruntkowski 954b70c79b
Run SiteLocker right after site ownership transfer (#3684)
* Run SiteLocker right after site ownership transfer

* Put site unlocking behind full build flag

* Tag regression tests as full build only
3 years ago
dependabot[bot] fae6c07fad
Bump oban from 2.17.1 to 2.17.2 (#3685)
Bumps [oban](https://github.com/sorentwo/oban) from 2.17.1 to 2.17.2.
- [Release notes](https://github.com/sorentwo/oban/releases)
- [Changelog](https://github.com/sorentwo/oban/blob/v2.17.2/CHANGELOG.md)
- [Commits](https://github.com/sorentwo/oban/compare/v2.17.1...v2.17.2)

---
updated-dependencies:
- dependency-name: oban
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
hq1 63dec378c2
Remove `connection` and `OAuther` dependencies (#3681)
* Remove `connection` dependency

https://github.com/elixir-ecto/connection

* Remove OAuther dependency

Some remnant from https://github.com/plausible/analytics/pull/27/files
3 years ago
hq1 32456d4348
Upgrade stack (deps, elixir 1.16, OTP 26.2.1) (#3678)
* Bump asdf erlang & elixir

* Bump erlang/elixir in the dockerfile

* Remove Oban.Stager config as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.14.md

* Configure Oban for tests as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.14.md

* Mark sampling hint clause with unsafe fragment

Any other/recommended way to do that? cc @ruslandoga

* Address String.slice/2 deprecation

* Update deps

* Address 0.0 matching warning

* Make funnel settings work

although this is probably not the best way to do it.
Needs revisiting, not sure what broke it - could not find
any breaking changes in related dependencies.

* Migrate oban as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.17.md

* Update credo

* Bump CI cache

* Use `Bypass.pass` to avoid exit shutdown message
3 years ago
hq1 c1a1d697a4
Partially revert #3661 - just keep the real errors wrapped, but don't display anything to the user (#3677)
* Revert "Remove unused RocketIcon"

This reverts commit c5e8d0c172.

* Revert "Display either hash or actual error message"

This reverts commit 0c091ab35f.

* Revert "Use ApiErrorNotice in funnels"

This reverts commit 5929de248e.

* Revert "Don't render "No data yet" when there's a NetworkError for example"

This reverts commit 70bee07632.

* Revert "Show the sinking shuttle notice whenever an API error occurs"

This reverts commit 9a62c8af2b.

* Revert "Add Hahash dependency"

This reverts commit b94207ea0a.

* Remove support hash
3 years ago
RobertJoonas 403f559b35
Limit Custom Properties access in Stats API (#3670)
* add new function and doc for allowed_props

* limit props access in Stats API queries

* use dot syntax instead
3 years ago
hq1 24a8aa2821
Reapply sentry context (#3675)
* Reapply "Sentry context in live views (#3672)"

This reverts commit 5449fead160064b8a0081c458cc5dcd34399eb0b.

* Make sure `:selection_made` is handled in `GoalSettings.Form`

That was a bit unepexcted.. normally `handle_info` is injected
by the LiveView use macro and it discards any message gracefully.
After switching to `use PlausibleWeb, :live_view` we're also
using `PlausibleWeb.Live.Flash` that happens to inject its own receive
clause for closing the flash. Which then renders the original,
overridable, `handle_info` catch-all obsolete.

* Update LV SentryContext only on connected sockets

(first mount already has the right context coming from Sentry plug)

* Make sure Live.ChoosePlan passes `current_user_id` session key
3 years ago
hq1 c87b165aef
Revert "Sentry context in live views (#3672)" (#3673)
This reverts commit 9bb2dc00d0.
3 years ago
hq1 9bb2dc00d0
Sentry context in live views (#3672)
* Add common LiveView macro to PlausibleWeb

* Keep peer data, URI and UA in /live websocket metadata

* Use new PlausibleWeb macro in existing LiveViews

* Implement adding some basic Sentry context `on_mount`

* Format

* Use macro in Live.FunnelSettings

* Update FunnelSettings.Form
3 years ago
Adrian Gruntkowski aa139a73b8
Fix calendar picker for comparisons on small screens (#3669) 3 years ago
Adrian Gruntkowski 1021942863
Fix props availability check when listing stats filters in dropdown (#3671) 3 years ago
hq1 dc8210dd84
Use tags rather than context to annotate support hashes in Sentry (#3668) 3 years ago
hq1 9cb44291f7
Rate limit e-mail changes (#3667) 3 years ago
hq1 5c8f39ac4c
Prevent crashes on ambiguity when casting timezones (#3663)
* Fall back to UTC when timezone gap found

* bugfix 1

* bugfix 2

* Apply safe tz conversion to all cases where it's done on a ts other than now

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
hq1 72b4e05bbf
Always enable e-mail verification on full build (#3666)
* Always enable e-mail verification on full build

* s/change/set

* Update lib/plausible/auth/user.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
hq1 4569ac09c1
Don't leak internal server errors, use support hash (#3661)
* Add Hahash dependency

* Don't leak internal server error details to the user

* Show the sinking shuttle notice whenever an API error occurs

* Don't render "No data yet" when there's a NetworkError for example

* Use ApiErrorNotice in funnels

* Display either hash or actual error message

The reason "internal-server-error" doesn't work well as a fallback
hash is that e.g. `NetworkError when attempting to fetch resource`
might be completely at client's fault. In such cases it's better
to display the whole thing still.

* Remove unused RocketIcon
3 years ago
dependabot[bot] a769447dcc
Bump actions/github-script from 6 to 7 (#3534)
Bumps [actions/github-script](https://github.com/actions/github-script) from 6 to 7.
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](https://github.com/actions/github-script/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/github-script
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
Vinicius Brasil 9e5d63ed96
Display premium features tabs on the dashboard (#3646)
* Display premium features tabs on the dashboard

This commit makes the funnels and props tab on the dashboard visible
even when the site owner's plan does not have access to that feature.
This is to raise awareness of those features, and if the site owner does
not want to see that tab, they can still click "Hide this report" to
hide it.

Previously, when the plan did not support a feature, the feature module
`enabled/1` function returned `false`, regardless of the
`sites.feature_enabled` toggle. This commit creates a new function
called `opted_out/1` to differentiate access from explicitly opting out
a feature.

* Remove unused data-conversions-enabled attribute

* DRY Plausible.Billing.Feature.check_availability/1 function

* Allow opting out features the user doesn't have access to

* add upgrade CTA to FeatureSetupNotice

* fix JS linting errors

* simplify notice.js

* fix behaviour when deleting funnels saved to localstorage

In case some other funnel exists, we will use that as the default
selected one. If not, a feature setup notice will be displayed again.

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
3 years ago
RobertJoonas 155a7a56a7
Fix goal filter inconsistency (#3658)
* remove unused code

* add e.name == "pageview" condition to pageview goals

This fixes the weird behavior where filtering by a pageview goal would
also return custom events and vice versa.

* update changelog
3 years ago
hq1 7755d4fd29
Lock traffic data backfill (#3648)
* WIP snapshot

* Don't assume that subscription and trial can't co-exist 🤷

* Don't look up subscription status if payment is overdue
3 years ago
hq1 b6a2acb57a
Update accept_traffic_until notifications (#3665)
* Update accept_traffic_until notifications:

  - extend has_stats to 48h
  - ensure the user is properly greeted

* Clarify the structure passed to email template function
3 years ago
Adrian Gruntkowski af9b34a1c6
Revert "Prevent scrollbar click to close modal (#3651)" (#3664)
This reverts commit 75a7ef5241.
3 years ago
hq1 21bbd3835a
Disallow funny business on timezone entry (#3662)
* Disallow funny business on timezone entry

* Add external API test
3 years ago
Marko Saric 7efa253e3f
Add a note about site settings being available on the locked screen (#3650)
* Add a note about site settings being available on the locked screen

some people are not aware that they can access site settings when the dashboard is locked. i've tried to add that to the note here as it should improve the user experience and minimize the number of people reaching out to support

* Adjust markup a bit

* Convert view to HEEx and apply predefined components

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
RobertJoonas 1342135908
Add `custom_props.csv` back to CSV export for Growth plans (#3656)
* restore special props access for growth plans

* format

* refactor if condition

Co-authored-by: hq1 <hq@mtod.org>

* format

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
hq1 8be9397199
Skip breakdown of imported pages with pageviews=0 (#3655)
* Skip breakdown of imported pages with pageviews=0

* Fixup

* Prove good impoted records are merged
3 years ago
hq1 73923404b0
Improve approaching accept_traffic_until e-mail (#3660)
* Improve approaching accept_traffic_until e-mail

* fixup

* format
3 years ago
Vinicius Brasil 75a7ef5241
Prevent scrollbar click to close modal (#3651)
This commit replaces the `mousedown` event for `click` to close a modal,
such as the detail dialog. This allows the scrollbar to be clicked
without firing an event, and right clicks too.

Closes #2872
3 years ago
ruslandoga af6b578dc5
format extra (#3657) 3 years ago
hq1 f755b20569
Lock traffic notifications (#3641)
* Update communication

* Remove an unreachable function (mistyped)

* [migration] Make accept_traffic_until a date

* Fix typo

* Set `accept_traffic_until` when creating a site

* Update sites `accept_traffic_until` on subscription change

* Add a note to yearly cancellation notification

* Rephrase annual e-mail for clarity

* Pass the small build test

* Add email notifications

* Fixup

* Implement `accept_traffic_until` notification worker

* Fixup - no need to test this for small build

* Update moduledoc

* Move moduletag

* s/sent_at/sent_on

* Use WHERE NOT EXISTS instead of LEFT JOIN

* Use upsert when tracking notifications sent

* Store sent marker before actually sending notification

* Prefer to keep `accept_traffic_until` on the user record

This gives us a single source of truth, addresses cases like
ownership transparently, simplifies the code and enables CRM toggles.
The only downside is that there's another join performed in the
Sites.Cache full refresh - in this case, small refreshes are
skipped - but this is fine, since the traffic will be let in
anyway.

* Expose `accepted_traffic_until` in the CRM

* Update lib/plausible/auth/user.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Preload owner in CRM

* Use the offset parameter in trial over e-mail contents

* Format

* Harden cache test

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Adrian Gruntkowski 66d4b3a966
Fix SiteLocker for case of user without trial expiry date but with active subscription (#3652) 3 years ago
RobertJoonas d2270f3c35
Small bugfix + refactor email reports (#3642)
* use more convenient testing functions

* do not display + sign with 0% change in emails

* Rename module/file/function names

before, `weekly_report` was also used for monthly reports and that was a
bit confusing to read in code.

* Refactor send_email_report.ex

This commit improves readability by refactoring the code into smaller
functions and reducing the number of arguments given to functions.

But more importantly, it stops making duplicate stats queries for every
email recipient by moving the queries out of the for loop.

* Refactor: move querying logic out of the worker module

and merge all stats information under a single `stats` assign.
3 years ago
Marko Saric 2bbc063b6c
adding data centers to the comparison 3 years ago
Cenk Kücük 147597cd98
Drop events when classified as datacenter IP (#3647) 3 years ago
Adrian Gruntkowski 9d97dc1912
Move limit enforcement to accepting site ownership transfer (#3612)
* Move limit enforcement to accepting site ownerhsip transfer

* enforce pageview limit on ownership transfer accept

* Refactor plan limit check logic

* Extract `ensure_can_take_ownership` to `Invitations` context and refactor

* Improve styling of exceeded limits notice in invitation dialog and disable button

* styling improvements to notice

* make transfer_ownership return transfer to self error

* do not allow transferring to user without active subscription WIP

* Add missing typespec and improve existing ones

* Fix formatting

* Explicitly label direct match on function argument for clarity

* Slightly refactor `CreateInvitation.bulk_transfer_ownership_direct`

* Exclude quota enforcement tests from small build test suite

* Remove unused return type from `invite_error()` union type

* Do not block plan upgrade when there's pending ownership transfer

* Don't block and only warn about missing features on transfer

* Remove `x-init` attribute used for debugging

* Add tests for `Quota.monthly_pageview_usage/2`

* Test and improve site admin ownership transfer actions

* Extend tests for `AcceptInvitation.transfer_ownership`

* Test transfer ownership controller level accept action error cases

* Test choosing plan by user without sites but with a pending ownership transfer

* Test invitation x-data in sites LV

* Remove sitelocker trigger in invitation acceptance code and simplify logic

* Add Quota test for `user.allow_next_upgrade_override` being set

* ignore pageview limit only when subscribing to plan

* Use sandbox Paddle instance for staging

* Use sandbox paddle key for staging and dev

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
3 years ago
hq1 1678fa10f4
Migrations for #3641 (#3644)
* Migrations for #3641

* Fix typo
3 years ago
Uku Taht 222de719ea
Fix dropdown highlighting for comparisons (#3643) 3 years ago
Uku Taht a3e365b01b
Remove onClick handler from details link (#3636) 3 years ago
RobertJoonas 22ecbe7bc7
Refactor: Split up the choose_plan LV code (#3637)
* move format_price to Plausible.Billing

* move PlausibleWeb.Components.Billing file to subfolder

* extract new Notice module

* rename test file and module name

* move growth_grandfathered notice to notice.ex

* extract a PlanBenefits module

* extract PlanBox component

* extract PageviewSlider component

* fix plan benefits text color
3 years ago
RobertJoonas 77ca5abbc8
fix theme applying based on the 'theme' query param (#3639) 3 years ago
hq1 9e783ebb41
Set CLICKHOUSE_MAX_BUFFER_SIZE_BYTES to 1M on dev (#3635)
Completes full `mix ecto.reset` in around 30s 🤘
3 years ago
dependabot[bot] 0a49511052
Bump eslint-plugin-playwright from 0.19.0 to 0.20.0 in /tracker (#3621)
Bumps [eslint-plugin-playwright](https://github.com/playwright-community/eslint-plugin-playwright) from 0.19.0 to 0.20.0.
- [Release notes](https://github.com/playwright-community/eslint-plugin-playwright/releases)
- [Changelog](https://github.com/playwright-community/eslint-plugin-playwright/blob/main/CHANGELOG.md)
- [Commits](https://github.com/playwright-community/eslint-plugin-playwright/compare/v0.19.0...v0.20.0)

---
updated-dependencies:
- dependency-name: eslint-plugin-playwright
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
Vinicius Brasil 7f51928338
Remove business tier feature flag (#3632)
* Remove business tier feature flag

This commit removes all code branches related to the business tier
feature flag, as we're not flipping this flag off anymore. It also
removes unused routes, e.g. /billing/change-plan and /billing/upgrade

* remove unused billing templates

* refactor with clause to case instead

* assert on the url in email tests

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
3 years ago
ruslandoga 6639d6af63
Flush Clickhouse buffers faster by encoding early (#3623)
* encode early

* rename: CLICKHOUSE_MAX_BUFFER_SIZE -> CLICKHOUSE_MAX_BUFFER_SIZE_BYTES

* deprecate CLICKHOUSE_MAX_BUFFER_SIZE

* cleanup

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
Uku Taht 17ba44e8db
Fix darkmode flash (#3625)
* Inline theme script to layout

* Extract theme_script as Heex component

* Remove unused function

* Reference assigns directly

* Clean up current user

* Clean up theme javascript

* Use new theme script in focus.html layout

* Use `assigns` instead of `@conn.assigns` in `focus.html.heex` template too

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
RobertJoonas ab54a66a43
Fix `class` in `styled_link` and remove existing uses of `class` (#3631)
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
hq1 5a01624f03
Fix link styling regression introduced via #3572 (#3629)
* Fix link styling regression introduced via #3572

* Justify external links in user dropdown

* Format

* Use assigns.new_tab to justify justification 🤡
3 years ago
hq1 8d85c38047
Add `:payment_required` policy to `GateKeeper` (#3628)
* Migration: Add `accept_traffic_until` to "sites"

* Drop traffic from sites where `accept_traffic_until` is passed
3 years ago
hq1 3f5eecf15f
Migration: Add `accept_traffic_until` to "sites" (#3627) 3 years ago
hq1 d161ad7667
Fix flaky test (#3626) 3 years ago
RobertJoonas 686ffa6ef3
Clean up legacy trials code + displaying limits (#3620)
* remove unused functionality

We can now safely delete all the logic around users who are on trial and
signed up *before* the business tiers release, since that's no longer
possible

* add monthly_pageview_limit fn clause that takes a user

* make Quota.site_limit return enterprise site limit

...not `:unlimited`, as we still need to display it
in the account settings.

* make `team_member_limit/1` return :unlimited on small_build

* improve team_member_usage/1 function doc

* stop displaying unlimited symbol in usage section

These unlimited limits include:

* `monthly_pageview_limit` for trials
* `team_member_limit` for old enterprise plans
* `site limit` for old accounts (before 2021-05-05)

* format

* small refactor case clause + move mod vars

* review suggestions
3 years ago
Uku Taht 677b4f2e79
Use dropdown component in /sites page (#3572)
* Use dropdown component in sites page

* Add x- attributes to globals

* Update tests

* Prevent default on click

* Fix compile warning

* Add conditional rendering back in

* Remove $id magic
3 years ago
ruslandoga 95e7a2b1f6
add session collapse order test (#3622) 3 years ago
RobertJoonas e85ee417ab
Prevent invited users without sites from subscribing (#3600)
* prevent invited users without sites from subscribing

* always setup a site in choose_plan_test context

* change trial_expiry_date=nil condition to no sites owned
3 years ago
hq1 8ba851d27f
Protect against custom props payloads breaking event insertions (#3617)
* Protect against custom props payloads breaking event insertions

* Use two traversals instead of three

* Reorganize invalid props filtering
3 years ago
hq1 b96319321c
Add Browser/Version breakdown to CSV (#3599)
* Add csv for browser versions

* add changelog

* Fixup - ensure the dashboard browser version breakdown still works

* Update CHANGELOG

---------

Co-authored-by: Ekaterina Krivich <ekaterinak@heathmont.net>
3 years ago
dependabot[bot] 906b06ae84
Bump eslint from 8.54.0 to 8.55.0 in /tracker (#3586)
Bumps [eslint](https://github.com/eslint/eslint) from 8.54.0 to 8.55.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/compare/v8.54.0...v8.55.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
Vinicius Brasil 45a763ab2b
Add data retention to the choose plan page (#3605)
* Add data retention field to plans

* Display data retention as benefit when choosing plan

* Split fields in two module attributes

* Remove extra whitespace

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
ruslandoga 26809bfd2e
Fix flaky rate_limit test (#3615) 3 years ago
hq1 7b5c20db17
Revert "experiment in improving throughput of write buffers (#3511)" (#3614)
This reverts commit 16bfb11701.
3 years ago
Adam Rutkowski 69579272d6 Revert "dump buffer on invalid insert (#3613)"
This reverts commit 7a4cafac16.
3 years ago
ruslandoga 7a4cafac16
dump buffer on invalid insert (#3613)
* dump buffer on invalid insert

* dump to PERSISTENT_CACHE_DIR

* fix warning

* ensure dir is not nil
3 years ago
ruslandoga 16bfb11701
experiment in improving throughput of write buffers (#3511)
* wip

* fewer length/1 calls

* encode early

* cleanup

* add micro benchmark

* extract types at compilation

* remove specs to make credo happier

* cleanup is_bounce

* cleanup :D

* add eprof to buffer insert+flush benchmark

* add ci/bench.yml

* rm state.schema from write buffer

* rm noisy logs

* rm benches

* add :source to insert

* make CLICKHOUSE_MAX_BUFFER_SIZE mean bytes

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
Vinicius Brasil e4230db2d9
Improve subscription status checking code (#3598)
* Improve subscription status checking code

This commit improves the subscription status checking in code, and
creates convinience functions to work with it, including nil-checking
and in?/2 function.

* Change in?/2 to macro
3 years ago
Adrian Gruntkowski 12f786810c
Remove `two_factor` feature flag (#3597)
* Remove `two_factor` feature flag

* Avoid compilation warnings in small build
3 years ago
ruslandoga 0eedf9aa98
rm Hammer (#3571) 3 years ago
Marko Saric 0b00762591
Changes to the emails as discussed (#3540)
* Update over_limit.html.eex

* Update dashboard_locked.html.eex

* Update dashboard_locked.html.eex

* Update over_limit.html.eex

* Update dashboard_locked.html.eex

* fix tests

* stop querying owned_site_ids three times

... when querying for billing cycles. Adds an optional `owned_site_ids`
argument to the `usage_cycle` function.

* add penultimate billing cycle info to emails

This commit also refactors some code and adds unit tests to email templates

* use delimit_integer instead of large_number_format

... to display usage with exact numbers such as 1,099,999 instead of 1M

* add penultimate cycle date ranges and linebreaks

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
3 years ago
hq1 615b6aef7d
Plugins API exentsions (custom props, bulk goal delete, goal creation => ListResponse always) (#3593)
* End polymorphic response in goals create

Being part of the v3 spec, this isn't well
(or at all) supported by OpenAPI generators.

Always respond with `Goal.ListResponse`

* Implement `PUT /custom_props`

* Implement bulk `DELETE /goals`

* Expose API for (bulk-)disabling custom props

* Add controller typespecs

* Delegate list wrapping to `Plausible.API.*`
3 years ago
Adrian Gruntkowski da0fa6c355
Implement UI for 2FA setup and verification (#3541)
* Add 2FA actions to `AuthController`

* Hook up new `AuthController` actions to router

* Add `qr_code` to project dependencies

* Implement generic `qr_code` component rendering SVG QR code from text

* Implement enabled and disabled 2FA setting state in user settings view

* Implement view for initiating 2FA setup

* Implement view for verifying 2FA setup

* Implement view for rendering generated 2FA recovery codes

* Implement view for verifying 2FA code

* Implement view for verifying 2FA recovery code

* Improve `input_with_clipboard` component

* Improve view for initiating 2FA setup

* Improve verify 2FA setup view

* Implement `verify_2fa_input` component

* Improve view for verifying 2FA setup

* Improve view rendering generated 2FA recovery codes

* Use `verify_2fa_input` component in verify 2FA view

* Do not render PA contact on self-hosted instances

* Improve flash message phrasing on generated recovery codes

* Add byline with a warning to disable 2FA modal

* Extract modal to component and move 2FA components to dedicated module

* First pass on loading state for "generate new codes"

* Adjust modal button logic

* Fix button in verify_2fa_input component

* Use button component in activate view

* Implement wait states for recovery code related actions properly

* Apply rate limiting to 2FA verification

* Log failed 2FA code input attempts

* Add ability to trust device and skip 2FA for 30 days

* Improve styling in dark mode

* Fix waiting state under Chrome and Safari

* Delete trust cookie when disabling 2FA

* Put 2FA behind a feature flag

* Extract 2FA cookie deletion

* ff fixup

* Improve session management during 2FA login

* Extract part of 2FA controller logic to a separate module and clean up a bit

* Clear 2FA user session when rate limit hit

* Add id to form in verify 2FA setup view

* Add controller tests for 2FA actions and login action

* Update CHANGELOG.md

* Use `full_build?()` instead of `@is_selfhost` removed after rebase

* Update `Auth.TOTP` moduledoc

* Add TOTP token management and make `TOTP.enable` more test-friendly

* Use TOTP token for device trust feature

* Use zero-deps `eqrcode` instead of deps-heavy `qr_code`

* Improve flash messages copy

Co-authored-by: hq1 <hq@mtod.org>

* Make one more copy improvement

Co-authored-by: hq1 <hq@mtod.org>

* Fix copy in remaining spots

* Change redirect after login to accept URLs from #3560 (h/t @aerosol)

* Add tests checking handling login_dest on login and 2FA verification

* Fix regression in email activation form submit button behavior

* Rename `PlausibleWeb.TwoFactor` -> `PlausibleWeb.TwoFactor.Session`

* Move `qr_code` component under `Components.TwoFactor`

* Set domain and secure options for new cookies

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
RobertJoonas 4566e6b530
New admin route for displaying usage (#3577)
* add a new crm usage route for admins

* add a test for admin route authorization

* add full_build_only tag
3 years ago
RobertJoonas 7212d336ae
add migration (#3590) 3 years ago
Adrian Gruntkowski cc18423369
Add `totp_token` string field to `users` (#3592) 3 years ago
Adrian Gruntkowski af363bf294 Remove test which is no longer conformant with current biz plan logic 3 years ago
Adrian Gruntkowski d42c6927d5
Use autocomplete=new-password for shared link password field (#3589) 3 years ago
Uku Taht 032823e112
Add last_bill_date to new subscriptions (#3588)
* Add last_bill_date to new subscriptions

* Remove leftover test code

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
3 years ago
Uku Taht 44d71c8c0e
Fix domains that start with UTF character (#3560)
* Avoid redirect in site settings

* Fix unicode in SiteController existing tests

* Fix various tests

* Add CHANGELOG

* Make sure test site is example.com

* Use Route helpers in site_controller

* Fix UTF redirect in change domain submit action

* Fix UTF site domain in reset stats action
3 years ago
hq1 c4eea9f8d5
`s/warn/warning` (`warn` is deprecated) (#3587) 3 years ago
hq1 74675f0cbc
Move dogfooding to full build (#3576)
* Move dogfooding to `extra/`

* Remove unused view functions
3 years ago
dependabot[bot] 466958ea60
Bump eslint-plugin-playwright from 0.18.0 to 0.19.0 in /tracker (#3578)
Bumps [eslint-plugin-playwright](https://github.com/playwright-community/eslint-plugin-playwright) from 0.18.0 to 0.19.0.
- [Release notes](https://github.com/playwright-community/eslint-plugin-playwright/releases)
- [Changelog](https://github.com/playwright-community/eslint-plugin-playwright/blob/main/CHANGELOG.md)
- [Commits](https://github.com/playwright-community/eslint-plugin-playwright/compare/v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: eslint-plugin-playwright
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
Vinicius Brasil 5e6033e261
Move input range bubble to JavaScript (#3569)
* Move input range bubble to JavaScript

This commit switches the input range bubble on the choose plan page from
LiveView to JavaScript. The reason for this change is the input range
is a regular HTML input rendered by the browser, not LV, therefore
bubble was not in sync when sliding the input.

* Apply suggestions from code review
3 years ago
hq1 3905e27d40
Pin https://github.com/docker/metadata-action/ (#3575)
Ref: https://github.com/docker/metadata-action/pull/352#issuecomment-1833733738
3 years ago
hq1 7bf1e2a6ed
Reapply "Define a better monthly pageview usage (#3564)" (#3574)
This reverts commit c739b8878d.
3 years ago
hq1 5278c23965
Revert and extract migration (#3573)
* Revert "Define a better monthly pageview usage (#3564)"

This reverts commit 57188a402a.

* Extract migration from 57188a402a/priv/repo/migrations/20231129103158_add_allow_next_upgrade_override_to_users.exs
3 years ago
RobertJoonas 57188a402a
Define a better monthly pageview usage (#3564)
* refactor asking for the monthly pageview usage

* add tests for usage and limits section in account settings

* display pageview usage per billing cycle for active subscribers

* disable cycle tabs if no usage

* make current billing cycle whole

...instead of capping it at today's date

* run queries for different cycles concurrently

* fix linebreak bug

* add calculate usage action into CRM

* change some names of assigns

* block subscribing to a plan by pageview usage

Depending on whether the customer has already subscribed or not, checking
their pageview usage is different:

* If they're not subscribed yet, we allow them to subscribe to a plan If
  it their last 30 days usage does not exceed the plan pageview limit by
  more than 15% (30% for when subscribing to a 10k plan)

* For existing subscribers, we'll use the exact same mechanism that we're
  using for locking sites - the last two billing cycles usage. If both
  cycles exceed the plan limit by more than 10% - we don't allow them to
  subscribe to the plan

* apply credo suggestion

* prevent highlight bar overflow

* move disabled classes to button element

* optimize for darkmode

* unify link and text styling on the same horizontal line

'Upgrade' & 'Update billing details' links + billing interval text were
positioned on the same line. The font size was similar, but not the same

* improve exceeded_limits function readability

* Refactor some tests and remove code duplication

* override allow upgrade when limits exceeded

In cases where limits are exceeded, we can set the boolean flag
`allow_next_upgrade_override` to `true` in the CRM. This will allow
the user to upgrade to any plan they want. After they've upgraded or
changed their plan - the flag will automatically reset to `false`.

* only apply upgrade override for exceeded pageview limit

* fix tests on the CI

* make current_cycle usage always displayed by default

* make pageview allowance margin more clear

* add comment
3 years ago
hq1 bd7deb5631
Update dockerfile for build options (#3567)
* Update dockerfile for build options

* Start permanent on small

* Default to small

* Default to small fixup
3 years ago
Vinicius Brasil 8dfaad56f0
Match choose plan styling with landing page (#3568)
* Match choose page styling with landing page

Related: https://github.com/plausible/website/pull/507

* Fix tests

* Apply suggestions from code review
3 years ago
dependabot[bot] 736baa25d8
Bump @playwright/test from 1.40.0 to 1.40.1 in /tracker (#3565)
Bumps [@playwright/test](https://github.com/microsoft/playwright) from 1.40.0 to 1.40.1.
- [Release notes](https://github.com/microsoft/playwright/releases)
- [Commits](https://github.com/microsoft/playwright/compare/v1.40.0...v1.40.1)

---
updated-dependencies:
- dependency-name: "@playwright/test"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
hq1 a4b9c3b8ba
Remove custom domains support + update build options (#3559)
* Disable super-admin checks on small build

* Mute a test writing to stdout

* Move sampling outside of small build

* Convert waiting_first_pageview to heex and stop relying on env vars

* Set site limit unlimited on small build

* Stop relying on app env to get trial expiry

* Remove custom domains - including migration

* Remove is_selfhosted from layout view

* Quota fixup

* Stop relying on app env for self hosted registration

* Stop relying on app env for pass reset success

* Apply on_trial? check only on full build

* Update templates relying on app env

* Adjusts auth controller tests for small build

* Trial fixup

* Fixup

* Stop relying on app env

* Rest of the fsckn owl

* Update typespecs

* Fix dialyzer warning

* Remove unused module

* Credo + format

* GeoIP is not, for full build

* Use `small_build?()` where applicable

* Implement bypassing FirstLaunchPlug without insertions

* Get Marko's patch de58a18a85

* Test is-dbip=false presence

* Fix typespec

* Remove future hardcodes

* Handle `nil` from `Plausible.Geo.database_type()`

* Remove XXX marker

* Use one typespec for two clauses

* Introduce `MIX_ENV=small_dev`

* Revert "Use one typespec for two clauses"

This reverts commit 8d8cd21764.
3 years ago
Adrian Gruntkowski 86b9d1680d
Simplify TOTP core logic and add email notifications (#3563)
* Modify API of `Auth.TOTP` to work with simplified flow

* Send email when 2FA is enabled and disabled

* Add tests for `initiated?/1`

* Add tests for email submission and improve recipient address composition

* Fix email tags

* Rename email templates to HEEx

* Fix formatting in email templates
3 years ago
RobertJoonas ff2c3346d2
Bugfix: Allow breakdown by internally used prop keys for Growth plans (#3562)
* fix bug - allow internally used prop key breakdown for Growth plans

* use case instead of with
3 years ago
Marko Saric be48ca2826
Update SECURITY.md 3 years ago
Marko Saric de58a18a85
Keeping footers consistent 3 years ago
Marko Saric 276461e4ea
Comparison updated to keep it consistent with the site 3 years ago
hq1 0f577a998c
Update credo config + extras (#3547)
* Disable CyclomaticComplexity checks

* Move AuhtorizeSitesAPI plug under extra
3 years ago
hq1 88e1d9dc28
Small build updates (#3546)
* Sites API

* Extract Revenue react api helpers

* !fixup

* Extract JS Money module to /extra

* Extract Revenue full build extras (tests pass for full)

* Update MIX_ENV=small mix test

* Remove dead code

* Add moduledocs

* Add credo config

* Trick dialyzer

* DRY revenue metrics

* Use more concise version of on_full_build macro

* Disable credo check
3 years ago
Vinicius Brasil af0b97e68a
Move Mix.Task.DumpPlans to Plausible.Release (#3544)
This commit moves the dump_plans Mix task to the Plausible.Release
module so it can run in production.

Usage: `./bin/plausible eval Plausible.Release.dump_plans`
3 years ago
dependabot[bot] f01b76cc41
Bump eslint from 8.53.0 to 8.54.0 in /tracker (#3533)
Bumps [eslint](https://github.com/eslint/eslint) from 8.53.0 to 8.54.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/compare/v8.53.0...v8.54.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
Vinicius Brasil b35096bbc8
Dump plan information to PostgreSQL (#3543)
* Use Ecto.Schema for casting plans from JSON files

* Dump plans to internal database table
3 years ago
Adrian Gruntkowski 65cc8980e0
Implement core logic for TOTP support (#3525)
* Add `nimble_totp`, `cloak` and `cloak_ecto` to project dependencies

* Setup Cloak-based secrets vault and create a dedicated Ecto type

* Add `totp_enabled|secret|last_used_at` fields to `User` schema

* Implement schema and stateless logic for TOTP recovery codes

* Implement core logic of TOTP auth

* Fix typos and improve style of doc comments

Co-authored-by: hq1 <hq@mtod.org>

* Fix moduledoc alignment

* Use more compact conditional expression

Co-authored-by: hq1 <hq@mtod.org>

* Disambiguate `I` as `7` when generating recovery codes (h/t @hq1)

* Fix a typo in runtime config error message

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
hq1 819449f19a
Include extra in docker image (#3537) 3 years ago
hq1 b9ec38038c
Add small build option (#3536)
* Update applications

* Clone community config

* Move modules to experimental dir

* Update runtime config

* Apply first set of compile-time conditionals

* Move funnel schemas to experimental

* Make funnel schema-less build compile

* Use experimental/lib for elixir code

* Move JS funnels to experimental

* Clean up conditional rendering

* Tidy up the pipeline

* Make two builds pass tests without warnings

* Reuse existing dotenvs

* Do a bunch of renames

* Clean up naming

* Run secondary CI

* Update router

* Remove RewriteFunnelDupes migration

Tests were disabled already and it was a one-off shot

* Fixup quota mixins

* Add moduledoc

* Change MIX_ENV for seconary test run

* Skip crm on small

* !fixup

* Exclude flags pipeline

* Update lib/plausible_web/controllers/stats_controller.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Adrian Gruntkowski 2512f3cbe2
Create migration adding TOTP fields to `users` and recovery codes table (#3535) 3 years ago
Adrian Gruntkowski 9b0a6e6e7d
Improve LiveView flash messages (#3523)
* Cancel existing flash timer before setting a new one

* Abstract flash logic across live views

* Render flash messages one after another
3 years ago
ruslandoga 4586e091a5
Update PromEx to fix a memory leak in benchmarks (#3532) 3 years ago
RobertJoonas 02a1271ee4
Bugfix site transfers (#3531)
* Bugfix: allow ownership transfers when premium features enabled but not used

Fields like `props_enabled` and `funnels_enabled` are true by default,
and these fields do not indicate whether the user/site is actually using
these features or not.

* allow site transfers if they will be at limit after transfer

* small refactor
3 years ago
Vinicius Brasil 0175158e81
List only premium features on the CRM (#3530) 3 years ago
Cenk Kücük f4d7ba70e8
drop IPs classified as datacenter (#3522)
* drop IPs classified as datacenter

* process event and increment counter instead of dropping
3 years ago
dependabot[bot] a2c9ba191e
Bump @playwright/test from 1.39.0 to 1.40.0 in /tracker (#3528)
Bumps [@playwright/test](https://github.com/microsoft/playwright) from 1.39.0 to 1.40.0.
- [Release notes](https://github.com/microsoft/playwright/releases)
- [Commits](https://github.com/microsoft/playwright/compare/v1.39.0...v1.40.0)

---
updated-dependencies:
- dependency-name: "@playwright/test"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3 years ago
Márton Salomváry 555eb25d20
Allow custom event timeseries in stats API (#3505)
* Allow custom event timeseries in stats API

* Fix linting error

---------

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
3 years ago
Vinicius Brasil cfaa5be8f4
Configurable limits for enterprise plans (#3527) 3 years ago
RobertJoonas d66322e12d
Limit grandfathering to *active* subscribers and trials (#3524)
* refactor asserting plan generation in plans_test.exs

* stop grandfathering old expired trials

For users who registered before the business tiers release, we want to
offer a chance to subscribe to a grandfathered plan. However, if they
let their trial expire and don't subscribe in the next 10 days, they'll
lose that opportunity.

* stop grandfathering expired subscriptions

* remove default title and icon from Generic.notice

* fix bug with dismissable notice

classList is null when dismissable_id is not given

* alias Plausible.Auth.User

* Refactor Generic.notice component

Make it easy to apply different colors

* move subscription_cancelled_notice across the app

And remove from user settings > subscription box. Also, include a note
about losing grandfathered status when letting the subscription expire.

* allow full width in Generic.notice

* use Generic.notice for subscription_past_due_notice

* use Generic.notice for subscription_paused_notice

* prevent two notices clashing into each other with gap-y-2

* define attrs for phx components

* optimize for light mode

* make subscription cancelled notice dismissable

but if it's dismiss, show it in the place where it was before in the
account settings > subscription box

* make function private

* replace function doc with regular comment to avoid compile warning

* use array for classnames

Co-authored-by: Vinicius Brasil <vini@hey.com>

* fix typos in function doc

---------

Co-authored-by: Vinicius Brasil <vini@hey.com>
3 years ago
ruslandoga 13055aafc0
use windows in 'time on page' queries (#3446)
* use windows in 'time on page' query

* add imported time on page back

* join imported_pages instead of extra query

* don't forget aggregated time on page!

* eh?

* no need for separate window clause

* use dynamic_filter_condition in aggregate_time_on_page

* it's avg, not total

* fix dynamic_filter_condition call

* tests pass

* fewer changes

* fewer changes + feature flag

* base quickfix

* fewer changes

* fewer changes?

* fewer changes!

* replace coalesce with if(empty(),etc)

* make window_aggregate_time_on_page return same result as neighbor_

* breakdown as well

* add failing test

* more info

* more info

* format example

* add failing aggregate.ex test

* fix breakdown.ex windowing time_on_page calculation

* fix aggregate.ex windowing time_on_page calculation
3 years ago

@ -0,0 +1,205 @@
# This file contains the configuration for Credo and you are probably reading
# this after creating it with `mix credo.gen.config`.
#
# If you find anything wrong or unclear in this file, please report an
# issue on GitHub: https://github.com/rrrene/credo/issues
#
%{
#
# You can have as many configs as you like in the `configs:` field.
configs: [
%{
#
# Run any config using `mix credo -C <name>`. If no config name is given
# "default" is used.
#
name: "default",
#
# These are the files included in the analysis:
files: %{
#
# You can give explicit globs or simply directories.
# In the latter case `**/*.{ex,exs}` will be used.
#
included: [
"lib/",
"test/",
"extra/"
],
excluded: [~r"/_build/", ~r"/deps/", ~r"/node_modules/"]
},
#
# Load and configure plugins here:
#
plugins: [],
#
# If you create your own checks, you must specify the source files for
# them here, so they can be loaded by Credo before running the analysis.
#
requires: [],
#
# If you want to enforce a style guide and need a more traditional linting
# experience, you can change `strict` to `true` below:
#
strict: false,
#
# To modify the timeout for parsing files, change this value:
#
parse_timeout: 5000,
#
# If you want to use uncolored output by default, you can change `color`
# to `false` below:
#
color: true,
#
# You can customize the parameters of any check by adding a second element
# to the tuple.
#
# To disable a check put `false` as second element:
#
# {Credo.Check.Design.DuplicatedCode, false}
#
checks: %{
enabled: [
#
## Consistency Checks
#
{Credo.Check.Consistency.ExceptionNames, []},
{Credo.Check.Consistency.LineEndings, []},
{Credo.Check.Consistency.ParameterPatternMatching, []},
{Credo.Check.Consistency.SpaceAroundOperators, []},
{Credo.Check.Consistency.SpaceInParentheses, []},
{Credo.Check.Consistency.TabsOrSpaces, []},
#
## Design Checks
#
# You can customize the priority of any check
# Priority values are: `low, normal, high, higher`
#
{Credo.Check.Design.AliasUsage,
[priority: :low, if_nested_deeper_than: 2, if_called_more_often_than: 0]},
# You can also customize the exit_status of each check.
# If you don't want TODO comments to cause `mix credo` to fail, just
# set this value to 0 (zero).
#
{Credo.Check.Design.TagTODO, [exit_status: 2]},
{Credo.Check.Design.TagFIXME, []},
#
## Readability Checks
#
{Credo.Check.Readability.AliasOrder, []},
{Credo.Check.Readability.FunctionNames, []},
{Credo.Check.Readability.LargeNumbers, []},
{Credo.Check.Readability.MaxLineLength, [priority: :low, max_length: 120]},
{Credo.Check.Readability.ModuleAttributeNames, []},
{Credo.Check.Readability.ModuleDoc, []},
{Credo.Check.Readability.ModuleNames, []},
{Credo.Check.Readability.ParenthesesInCondition, []},
{Credo.Check.Readability.ParenthesesOnZeroArityDefs, []},
{Credo.Check.Readability.PipeIntoAnonymousFunctions, []},
{Credo.Check.Readability.PredicateFunctionNames, []},
{Credo.Check.Readability.PreferImplicitTry, []},
{Credo.Check.Readability.RedundantBlankLines, []},
{Credo.Check.Readability.Semicolons, []},
{Credo.Check.Readability.SpaceAfterCommas, []},
{Credo.Check.Readability.StringSigils, []},
{Credo.Check.Readability.TrailingBlankLine, []},
{Credo.Check.Readability.TrailingWhiteSpace, []},
{Credo.Check.Readability.UnnecessaryAliasExpansion, []},
{Credo.Check.Readability.VariableNames, []},
{Credo.Check.Readability.WithSingleClause, []},
#
## Refactoring Opportunities
#
{Credo.Check.Refactor.Apply, []},
{Credo.Check.Refactor.CondStatements, []},
{Credo.Check.Refactor.CyclomaticComplexity, false},
{Credo.Check.Refactor.FunctionArity, []},
{Credo.Check.Refactor.LongQuoteBlocks, []},
{Credo.Check.Refactor.MatchInCondition, []},
{Credo.Check.Refactor.MapJoin, []},
{Credo.Check.Refactor.NegatedConditionsInUnless, []},
{Credo.Check.Refactor.NegatedConditionsWithElse, false},
{Credo.Check.Refactor.Nesting, []},
{Credo.Check.Refactor.UnlessWithElse, []},
{Credo.Check.Refactor.WithClauses, []},
{Credo.Check.Refactor.FilterFilter, []},
{Credo.Check.Refactor.RejectReject, []},
{Credo.Check.Refactor.RedundantWithClauseResult, []},
#
## Warnings
#
{Credo.Check.Warning.ApplicationConfigInModuleAttribute, []},
{Credo.Check.Warning.BoolOperationOnSameValues, []},
{Credo.Check.Warning.ExpensiveEmptyEnumCheck, []},
{Credo.Check.Warning.IExPry, []},
{Credo.Check.Warning.IoInspect, []},
{Credo.Check.Warning.OperationOnSameValues, []},
{Credo.Check.Warning.OperationWithConstantResult, []},
{Credo.Check.Warning.RaiseInsideRescue, []},
{Credo.Check.Warning.SpecWithStruct, []},
{Credo.Check.Warning.WrongTestFileExtension, []},
{Credo.Check.Warning.UnusedEnumOperation, []},
{Credo.Check.Warning.UnusedFileOperation, []},
{Credo.Check.Warning.UnusedKeywordOperation, []},
{Credo.Check.Warning.UnusedListOperation, []},
{Credo.Check.Warning.UnusedPathOperation, []},
{Credo.Check.Warning.UnusedRegexOperation, []},
{Credo.Check.Warning.UnusedStringOperation, []},
{Credo.Check.Warning.UnusedTupleOperation, []},
{Credo.Check.Warning.UnsafeExec, []}
],
disabled: [
#
# Checks scheduled for next check update (opt-in for now, just replace `false` with `[]`)
#
# Controversial and experimental checks (opt-in, just move the check to `:enabled`
# and be sure to use `mix credo --strict` to see low priority checks)
#
{Credo.Check.Consistency.MultiAliasImportRequireUse, []},
{Credo.Check.Consistency.UnusedVariableNames, []},
{Credo.Check.Design.DuplicatedCode, []},
{Credo.Check.Design.SkipTestWithoutComment, []},
{Credo.Check.Readability.AliasAs, []},
{Credo.Check.Readability.BlockPipe, []},
{Credo.Check.Readability.ImplTrue, []},
{Credo.Check.Readability.MultiAlias, []},
{Credo.Check.Readability.NestedFunctionCalls, []},
{Credo.Check.Readability.SeparateAliasRequire, []},
{Credo.Check.Readability.SingleFunctionToBlockPipe, []},
{Credo.Check.Readability.SinglePipe, []},
{Credo.Check.Readability.Specs, []},
{Credo.Check.Readability.StrictModuleLayout, []},
{Credo.Check.Readability.WithCustomTaggedTuple, []},
{Credo.Check.Refactor.ABCSize, []},
{Credo.Check.Refactor.AppendSingleItem, []},
{Credo.Check.Refactor.DoubleBooleanNegation, []},
{Credo.Check.Refactor.FilterReject, []},
{Credo.Check.Refactor.IoPuts, []},
{Credo.Check.Refactor.MapMap, []},
{Credo.Check.Refactor.ModuleDependencies, []},
{Credo.Check.Refactor.NegatedIsNil, []},
{Credo.Check.Refactor.PipeChainStart, []},
{Credo.Check.Refactor.RejectFilter, []},
{Credo.Check.Refactor.VariableRebinding, []},
{Credo.Check.Warning.LazyLogging, []},
{Credo.Check.Warning.LeakyEnvironment, []},
{Credo.Check.Warning.MapGetUnsafePass, []},
{Credo.Check.Warning.MixEnv, []},
{Credo.Check.Warning.UnsafeToAtom, []}
# {Credo.Check.Refactor.MapInto, []},
#
# Custom checks can be created using `mix credo.gen.check`.
#
]
}
}
]
}

@ -0,0 +1,73 @@
# This file excludes paths from the Docker build context.
#
# By default, Docker's build context includes all files (and folders) in the
# current directory. Even if a file isn't copied into the container it is still sent to
# the Docker daemon.
#
# There are multiple reasons to exclude files from the build context:
#
# 1. Prevent nested folders from being copied into the container (ex: exclude
# /assets/node_modules when copying /assets)
# 2. Reduce the size of the build context and improve build time (ex. /build, /deps, /doc)
# 3. Avoid sending files containing sensitive information
#
# More information on using .dockerignore is available here:
# https://docs.docker.com/engine/reference/builder/#dockerignore-file
.dockerignore
# Ignore git, but keep git HEAD and refs to access current commit hash if needed:
#
# $ cat .git/HEAD | awk '{print ".git/"$2}' | xargs cat
# d0b8727759e1e0e7aa3d41707d12376e373d5ecc
.git
!.git/HEAD
!.git/refs
# Common development/test artifacts
/cover/
/doc/
/test/
/tmp/
.elixir_ls
plausible-report.xml
.env
.idea
*.iml
*.log
*.code-workspace
.vscode
# Mix artifacts
/_build/
/deps/
*.ez
# Generated on crash by the VM
erl_crash.dump
# If NPM crashes, it generates a log, let's ignore it too.
npm-debug.log
# Static artifacts - These should be fetched and built inside the Docker image
/assets/node_modules/
/tracker/node_modules/
/priv/static/cache_manifest.json
/priv/static/css
/priv/static/js
/priv/version.json
# Auto-generated tracker files
/priv/tracker/js/*.js
# Dializer
/priv/plts/*.plt
/priv/plts/*.plt.hash
# Geolocation databases
/priv/geodb/*.mmdb
/priv/geodb/*.mmdb.gz
# Docker volumes
.clickhouse_db_vol*
plausible_db*

@ -2,5 +2,5 @@
plugins: [Phoenix.LiveView.HTMLFormatter],
import_deps: [:ecto, :ecto_sql, :phoenix],
subdirectories: ["priv/*/migrations"],
inputs: ["*.{heex,ex,exs}", "{config,lib,test}/**/*.{heex,ex,exs}", "priv/*/seeds.exs"]
inputs: ["*.{heex,ex,exs}", "{config,lib,test,extra}/**/*.{heex,ex,exs}", "priv/*/seeds.exs"]
]

@ -0,0 +1,67 @@
name: Build Private Images GHCR
on:
push:
branches: [master, stable]
tags: ['r*']
pull_request:
types: [synchronize, labeled]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
if: ${{ github.event_name == 'push' || contains(github.event.pull_request.labels.*.name, 'deploy-to-staging') }}
runs-on: buildjet-16vcpu-ubuntu-2204
permissions:
contents: read
packages: write
steps:
- name: Docker meta
id: meta
uses: docker/metadata-action@v5.0.0
with:
images: ghcr.io/plausible/analytics/ee
tags: |
type=ref,event=pr
type=ref,event=branch
type=ref,event=tag
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
id: docker_build
uses: docker/build-push-action@v5
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
MIX_ENV=prod
BUILD_METADATA=${{ steps.meta.outputs.json }}
- name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
- name: Notify team on failure
if: ${{ failure() }}
uses: fjogeleit/http-request-action@v1
with:
url: ${{ secrets.BUILD_NOTIFICATION_URL }}
method: 'POST'
customHeaders: '{"Content-Type": "application/json"}'
data: '{"content": "<a href=\"https://github.com/plausible/analytics/actions/workflows/build-private-images.yml\">Build failed</a>"}'

@ -5,7 +5,6 @@ on:
branches: [master, stable]
tags: ['r*']
pull_request:
branches: [master]
types: [synchronize, labeled]
concurrency:
@ -20,7 +19,7 @@ jobs:
steps:
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v5.0.0
with:
images: plausible/analytics-private
tags: |
@ -47,6 +46,7 @@ jobs:
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
MIX_ENV=prod
BUILD_METADATA=${{ steps.meta.outputs.json }}
- name: Image digest

@ -0,0 +1,68 @@
name: Build Public Images GHCR
on:
push:
tags: ['v*']
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: buildjet-16vcpu-ubuntu-2204
permissions:
contents: read
packages: write
steps:
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/plausible/community-edition
tags: |
type=semver,pattern={{version}},prefix=v
type=semver,pattern={{major}}.{{minor}},prefix=v
type=semver,pattern={{major}},prefix=v
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
id: docker_build
uses: docker/build-push-action@v5
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/amd64, linux/arm64
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
MIX_ENV=small
BUILD_METADATA=${{ steps.meta.outputs.json }}
ERL_FLAGS=+JPperf true
- name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
- name: Notify team on failure
if: ${{ failure() }}
uses: fjogeleit/http-request-action@v1
with:
url: ${{ secrets.BUILD_NOTIFICATION_URL }}
method: 'POST'
customHeaders: '{"Content-Type": "application/json"}'
data: '{"content": "<a href=\"https://github.com/plausible/analytics/actions/workflows/build-public-images.yml\">Build failed</a>"}'

@ -17,7 +17,7 @@ jobs:
id: meta
uses: docker/metadata-action@v5
with:
images: plausible/analytics
images: plausible/community-edition
tags: |
type=semver,pattern={{version}},prefix=v
type=semver,pattern={{major}}.{{minor}},prefix=v
@ -46,8 +46,9 @@ jobs:
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
MIX_ENV=small
BUILD_METADATA=${{ steps.meta.outputs.json }}
ERL_FLAGS=+JPperf true
ERL_FLAGS=+JMsingle true
- name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}

@ -1,27 +1,42 @@
name: Elixir CI
on:
push:
branches: [master, stable]
pull_request:
push:
branches: [master, stable]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
CACHE_VERSION: v7
PERSISTENT_CACHE_DIR: cached
jobs:
build:
name: Build and test
name: "Build and test (${{ matrix.mix_env }}, ${{ matrix.postgres_image }}${{ matrix.test_experimental_reduced_joins == '1' && ', experimental_reduced_joins' || '' }})"
runs-on: ubuntu-latest
strategy:
matrix:
postgres_version: [13, 14, 15]
mix_env: ['test', 'small_test']
postgres_image: ['postgres:16']
test_experimental_reduced_joins: ['0']
include:
- mix_env: 'test'
postgres_image: 'postgres:15'
test_experimental_reduced_joins: '0'
- mix_env: 'test'
postgres_image: 'postgres:16'
test_experimental_reduced_joins: '1'
env:
MIX_ENV: ${{ matrix.mix_env }}
TEST_EXPERIMENTAL_REDUCED_JOINS: ${{ matrix.test_experimental_reduced_joins }}
services:
postgres:
image: postgres:${{ matrix.postgres_version }}
image: ${{ matrix.postgres_image }}
ports:
- 5432:5432
env:
@ -31,7 +46,6 @@ jobs:
--health-interval 10s
--health-timeout 5s
--health-retries 5
clickhouse:
image: clickhouse/clickhouse-server:23.3.7.5-alpine
ports:
@ -42,41 +56,91 @@ jobs:
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: marocchino/tool-versions-action@v1
id: versions
- uses: erlef/setup-beam@v1
with:
elixir-version: ${{ steps.versions.outputs.elixir }}
otp-version: ${{ steps.versions.outputs.erlang }}
- uses: actions/cache@v4
with:
path: |
deps
_build
tracker/node_modules
priv/tracker/js
${{ env.PERSISTENT_CACHE_DIR }}
key: ${{ env.MIX_ENV }}-${{ env.CACHE_VERSION }}-${{ github.head_ref || github.ref }}-${{ hashFiles('**/mix.lock') }}
restore-keys: |
${{ env.MIX_ENV }}-${{ env.CACHE_VERSION }}-${{ github.head_ref || github.ref }}-
${{ env.MIX_ENV }}-${{ env.CACHE_VERSION }}-refs/heads/master-
- name: Check for changes in tracker/**
uses: dorny/paths-filter@v3
id: changes
with:
filters: |
tracker:
- 'tracker/**'
- name: Check if priv/tracker/js/plausible.js exists
run: |
if [ -f priv/tracker/js/plausible.js ]; then
echo "HAS_BUILT_TRACKER=true" >> $GITHUB_ENV
else
echo "HAS_BUILT_TRACKER=false" >> $GITHUB_ENV
fi
- run: npm install --prefix ./tracker
if: steps.changes.outputs.tracker == 'true' || env.HAS_BUILT_TRACKER == 'false'
- run: npm run deploy --prefix ./tracker
if: steps.changes.outputs.tracker == 'true' || env.HAS_BUILT_TRACKER == 'false'
- run: mix deps.get --only $MIX_ENV
- run: mix compile --warnings-as-errors --all-warnings
- run: mix do ecto.create, ecto.migrate
- run: mix run -e "Tzdata.ReleaseUpdater.poll_for_update"
- run: mix test --include slow --include minio --max-failures 1 --warnings-as-errors
if: env.MIX_ENV == 'test'
- run: mix test --include slow --max-failures 1 --warnings-as-errors
if: env.MIX_ENV == 'small_test'
static:
name: Static checks (format, credo, dialyzer)
env:
MIX_ENV: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Read .tool-versions
uses: marocchino/tool-versions-action@v1
- uses: marocchino/tool-versions-action@v1
id: versions
- name: Set up Elixir
uses: erlef/setup-beam@v1
- uses: erlef/setup-beam@v1
with:
elixir-version: ${{steps.versions.outputs.elixir}}
otp-version: ${{ steps.versions.outputs.erlang}}
- name: Restore dependencies cache
uses: buildjet/cache@v3
elixir-version: ${{ steps.versions.outputs.elixir }}
otp-version: ${{ steps.versions.outputs.erlang }}
- uses: actions/cache@v4
with:
path: |
deps
_build
priv/plts
key: ${{ runner.os }}-mix-v6-${{ hashFiles('**/mix.lock') }}
restore-keys: ${{ runner.os }}-mix-v6-
- name: Install dependencies
run: mix deps.get && npm install --prefix ./tracker
- name: Check Formatting
run: mix format --check-formatted
- name: Check Compile Warnings
run: mix compile --warnings-as-errors --all-warnings
- name: Generate tracker files
run: npm run deploy --prefix ./tracker
- name: Check Credo Warnings
run: mix credo diff --from-git-merge-base origin/master
- name: Run tests
run: mix test --include slow --max-failures 1 --warnings-as-errors
- name: Check Dialyzer
run: mix dialyzer
env:
MIX_ENV: test
key: static-${{ env.MIX_ENV }}-${{ env.CACHE_VERSION }}-${{ github.head_ref || github.ref }}-${{ hashFiles('**/mix.lock') }}
restore-keys: |
static-${{ env.MIX_ENV }}-${{ env.CACHE_VERSION }}-${{ github.head_ref || github.ref }}-
static-${{ env.MIX_ENV }}-${{ env.CACHE_VERSION }}-refs/heads/master-
- run: mix deps.get
- run: mix compile --warnings-as-errors --all-warnings
- run: mix format --check-formatted
- run: mix deps.unlock --check-unused
- run: mix credo diff --from-git-merge-base origin/master
- run: mix dialyzer

@ -0,0 +1,28 @@
name: Validate migrations
on:
pull_request:
paths:
- 'priv/repo/migrations/**'
- 'priv/ingest_repo/migrations/**'
jobs:
validate:
name: App code does not change at the same time
runs-on: ubuntu-latest
steps:
- uses: dorny/paths-filter@v3
id: changes
with:
list-files: json
filters: |
lib:
- 'lib/**'
- 'extra/**'
- 'config/**'
- if: steps.changes.outputs.lib == 'true'
run: |
echo "::error file=${{ fromJSON(steps.changes.outputs.lib_files)[0] }}::Code and migrations shouldn't be changed at the same time"
exit 1

@ -4,7 +4,6 @@ on:
push:
branches: [master, stable]
pull_request:
branches: [master, stable]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}

@ -35,7 +35,7 @@ jobs:
otp-version: ${{ steps.versions.outputs.erlang}}
- name: Restore Elixir dependencies cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: |
deps

@ -4,11 +4,9 @@ on:
push:
branches:
- master
paths:
- 'test/e2e/**.tf'
paths:
- 'test/e2e/**.tf'
pull_request:
branches:
- master
paths:
- 'test/e2e/**.tf'
@ -43,7 +41,7 @@ jobs:
- name: Terraform Init
id: init
run: terraform init
- name: Terraform Validate
id: validate
run: terraform validate -no-color
@ -54,7 +52,7 @@ jobs:
run: terraform plan -no-color
continue-on-error: true
- uses: actions/github-script@v6
- uses: actions/github-script@v7
if: github.event_name == 'pull_request'
env:
PLAN: "terraform\n${{ steps.plan.outputs.stdout }}"

4
.gitignore vendored

@ -19,6 +19,9 @@ erl_crash.dump
# Also ignore archive artifacts (built via "mix archive.build").
*.ez
# Temporary files, for example, from tests.
/tmp/
# Ignore package tarball (built via "mix hex.build").
plausible-*.tar
@ -55,6 +58,7 @@ plausible-report.xml
*.iml
*.log
*.code-workspace
.vscode
# Dializer
/priv/plts/*.plt

@ -1,3 +1,3 @@
erlang 26.1.2
elixir 1.15.7-otp-26
erlang 26.2.1
elixir 1.16.0-otp-26
nodejs 21.0.0

@ -1,9 +1,25 @@
# Changelog
All notable changes to this project will be documented in this file.
## Unreleased
### Added
- Pages Block List in Site Settings
- Add `conversion_rate` to Stats API Timeseries and on the main graph
- Add `total_conversions` and `conversion_rate` to `visitors.csv` in a goal-filtered CSV export
- Ability to display total conversions (with a goal filter) on the main graph
- Add `conversion_rate` to Stats API Timeseries and on the main graph
- Add `time_on_page` metric into the Stats API
- County Block List in Site Settings
- Query the `views_per_visit` metric based on imported data as well if possible
- Group `operating_system_versions` by `operating_system` in Stats API breakdown
- Add `operating_system_versions.csv` into the CSV export
- Display `Total visitors`, `Conversions`, and `CR` in the "Details" views of Countries, Regions and Cities (when filtering by a goal)
- Add `conversion_rate` to Regions and Cities reports (when filtering by a goal)
- Add the `conversion_rate` metric to Stats API Breakdown and Aggregate endpoints
- IP Block List in Site Settings
- Allow filtering with `contains`/`matches` operator for Sources, Browsers and Operating Systems.
- Allow filtering by multiple custom properties
- Wildcard and member filtering on the Stats API `event:goal` property
- Allow filtering with `contains`/`matches` operator for custom properties
- Add `referrers.csv` to CSV export
- Add a new Properties section in the dashboard to break down by custom properties
- Add `custom_props.csv` to CSV export (almost the same as the old `prop_breakdown.csv`, but has different column headers, and includes props for pageviews too, not only custom events)
@ -16,18 +32,38 @@ All notable changes to this project will be documented in this file.
- Add last 24h plots to /sites view
- Add site pinning to /sites view
- Add support for JSON logger, via LOG_FORMAT=json environment variable
- Add support for 2FA authentication
- Add 'browser_versions.csv' to CSV export
- Add `CLICKHOUSE_MAX_BUFFER_SIZE_BYTES` env var which defaults to `100000` (100KB)
- Add alternative SMTP adapter plausible/analytics#3654
- Add `EXTRA_CONFIG_PATH` env var to specify extra Elixir config plausible/analytics#3906
- Add restrictive `robots.txt` for self-hosted plausible/analytics#3905
### Removed
- Removed the nested custom event property breakdown UI when filtering by a goal in Goal Conversions
- Removed the `prop_names` returned in the Stats API `event:goal` breakdown response
- Removed the `prop-breakdown.csv` file from CSV export
- Deprecated `CLICKHOUSE_MAX_BUFFER_SIZE`
- Removed `/app/init-admin.sh` that was deprecated in v2.0.0 plausible/analytics#3903
- Remove `DISABLE_AUTH` deprecation warning plausible/analytics#3904
### Changed
- A visits `entry_page` and `exit_page` is only set and updated for pageviews, not custom events
- Limit the number of Goal Conversions shown on the dashboard and render a "Details" link when there are more entries to show
- Show Outbound Links / File Downloads / 404 Pages / Cloaked Links instead of Goal Conversions when filtering by the corresponding goal
- Require custom properties to be explicitly added from Site Settings > Custom Properties in order for them to show up on the dashboard
- GA/SC sections moved to new settings: Integrations
- Replace `CLICKHOUSE_MAX_BUFFER_SIZE` with `CLICKHOUSE_MAX_BUFFER_SIZE_BYTES`
- Validate metric isn't queried multiple times
### Fixed
- Creating many sites no longer leads to cookie overflow
- Ignore sessions without pageviews for `entry_page` and `exit_page` breakdowns
- Using `VersionedCollapsingMergeTree` to store visit data to avoid rare race conditions that led to wrong visit data being shown
- Fix `conversion_rate` metric in a `browser_versions` breakdown
- Calculate `conversion_rate` percentage change in the same way like `bounce_rate` (subtraction instead of division)
- Calculate `bounce_rate` percentage change in the Stats API in the same way as it's done in the dashboard
- Stop returning custom events in goal breakdown with a pageview goal filter and vice versa
- Only return `(none)` values in custom property breakdown for the first page (pagination) of results
- Fixed weekly/monthly e-mail report [rendering issues](https://github.com/plausible/analytics/issues/284)
- Fix [broken interval selection](https://github.com/plausible/analytics/issues/2982) in the all time view plausible/analytics#3110
@ -38,6 +74,13 @@ All notable changes to this project will be documented in this file.
- Update bot detection (matomo 6.1.4, ua_inspector 3.4.0)
- Improved the Goal Settings page (search, autcompletion etc.)
- Log mailer errors plausible/analytics#3336
- Allow custom event timeseries in stats API plausible/analytics#3505
- Fixes for sites with UTF characters in domain plausible/analytics#3560
- Fix crash when using special characters in filter plausible/analytics#3634
- Fix automatic scrolling to the bottom on the dashboard if previously selected properties tab plausible/analytics#3872
- Allow running the container with arbitrary UID plausible/analytics#2986
- Fix `width=manual` in embedded dashboards plausible/analytics#3910
- Fix URL escaping when pipes are used in UTM tags plausible/analytics#3930
## v2.0.0 - 2023-07-12

@ -2,10 +2,12 @@
# platform specific, it makes sense to build it in the docker
#### Builder
FROM hexpm/elixir:1.15.7-erlang-26.1.2-alpine-3.18.4 as buildcontainer
FROM hexpm/elixir:1.16.0-erlang-26.2.1-alpine-3.18.4 as buildcontainer
ARG MIX_ENV=small
# preparation
ENV MIX_ENV=prod
ENV MIX_ENV=$MIX_ENV
ENV NODE_ENV=production
ENV NODE_OPTIONS=--openssl-legacy-provider
@ -39,13 +41,13 @@ COPY assets ./assets
COPY tracker ./tracker
COPY priv ./priv
COPY lib ./lib
COPY extra ./extra
RUN npm run deploy --prefix ./tracker && \
mix assets.deploy && \
mix phx.digest priv/static && \
mix download_country_database && \
# https://hexdocs.pm/sentry/Sentry.Sources.html#module-source-code-storage
mix sentry_recompile
mix sentry.package_source_code
WORKDIR /app
COPY rel rel
@ -58,19 +60,18 @@ LABEL maintainer="plausible.io <hello@plausible.io>"
ARG BUILD_METADATA={}
ENV BUILD_METADATA=$BUILD_METADATA
ENV LANG=C.UTF-8
ARG MIX_ENV=small
ENV MIX_ENV=$MIX_ENV
RUN apk upgrade --no-cache
RUN adduser -S -H -u 999 -G nogroup plausible -g 'Plausible Analytics'
RUN apk upgrade --no-cache
RUN apk add --no-cache openssl ncurses libstdc++ libgcc ca-certificates
COPY ./rel/docker-entrypoint.sh /entrypoint.sh
RUN chmod a+x /entrypoint.sh && \
adduser -h /app -u 1000 -s /bin/sh -D plausibleuser
COPY --from=buildcontainer --chmod=a+rX /app/_build/${MIX_ENV}/rel/plausible /app
COPY --chmod=755 ./rel/docker-entrypoint.sh /entrypoint.sh
COPY --from=buildcontainer /app/_build/prod/rel/plausible /app
RUN chown -R plausibleuser:plausibleuser /app
USER plausibleuser
USER 999
WORKDIR /app
ENV LISTEN_IP=0.0.0.0
ENTRYPOINT ["/entrypoint.sh"]

@ -18,7 +18,7 @@ server: ## Start the web server
CH_FLAGS ?= --detach -p 8123:8123 -p 9000:9000 --ulimit nofile=262144:262144 --name plausible_clickhouse
clickhouse: ## Start a container with a recent version of clickhouse
docker run $(CH_FLAGS) --volume=$$PWD/.clickhouse_db_vol:/var/lib/clickhouse clickhouse/clickhouse-server:22.9-alpine
docker run $(CH_FLAGS) --volume=$$PWD/.clickhouse_db_vol:/var/lib/clickhouse clickhouse/clickhouse-server:latest-alpine
clickhouse-prod: ## Start a container with the same version of clickhouse as the one in prod
docker run $(CH_FLAGS) --volume=$$PWD/.clickhouse_db_vol_prod:/var/lib/clickhouse clickhouse/clickhouse-server:23.3.7.5-alpine
@ -36,3 +36,12 @@ postgres-prod: ## Start a container with the same version of postgres as the one
postgres-stop: ## Stop and remove the postgres container
docker stop plausible_db && docker rm plausible_db
minio: ## Start a transient container with a recent version of minio (s3)
docker run -d --rm -p 10000:10000 -p 10001:10001 --name plausible_minio minio/minio server /data --address ":10000" --console-address ":10001"
while ! docker exec plausible_minio mc alias set local http://localhost:10000 minioadmin minioadmin; do sleep 1; done
docker exec plausible_minio mc mb local/dev-exports
docker exec plausible_minio mc mb local/dev-imports
minio-stop: ## Stop and remove the minio container
docker stop plausible_minio

@ -15,7 +15,7 @@
<br /><br />
</p>
[Plausible Analytics](https://plausible.io/) is an easy to use, lightweight (< 1 KB), open source and privacy-friendly alternative to Google Analytics. It doesn’t use cookies and is fully compliant with GDPR, CCPA and PECR. You can self-host or have us run Plausible for you in the cloud. Here's [the live demo of our own website stats](https://plausible.io/plausible.io). Made and hosted in the EU 🇪🇺
[Plausible Analytics](https://plausible.io/) is an easy to use, lightweight (< 1 KB), open source and privacy-friendly alternative to Google Analytics. It doesn’t use cookies and is fully compliant with GDPR, CCPA and PECR. You can self-host Plausible Community Edition or have us manage Plausible Analytics for you in the cloud. Here's [the live demo of our own website stats](https://plausible.io/plausible.io). Made and hosted in the EU 🇪🇺
We are dedicated to making web analytics more privacy-friendly. Our mission is to reduce corporate surveillance by providing an alternative web analytics tool which doesn’t come from the AdTech world. We are completely independent and solely funded by our subscribers.
@ -23,7 +23,7 @@ We are dedicated to making web analytics more privacy-friendly. Our mission is t
## Why Plausible?
Here's what makes Plausible a great Google Analytics alternative and why we're trusted by 11,000+ paying subscribers to deliver their website and business insights:
Here's what makes Plausible a great Google Analytics alternative and why we're trusted by 12,000+ paying subscribers to deliver their website and business insights:
- **Clutter Free**: Plausible Analytics provides [simple web analytics](https://plausible.io/simple-web-analytics) and it cuts through the noise. No layers of menus, no need for custom reports. Get all the important insights on one single page. No training necessary.
- **GDPR/CCPA/PECR compliant**: Measure traffic, not individuals. No personal data or IP addresses are ever stored in our database. We don't use cookies or any other persistent identifiers. [Read more about our data policy](https://plausible.io/data-policy)
@ -49,31 +49,32 @@ We choose the subscription business model rather than the business model of surv
## Getting started with Plausible
The easiest way to get started with Plausible is with [our official managed service in the cloud](https://plausible.io/#pricing). It takes 2 minutes to start counting your stats with a worldwide CDN, high availability, backups, security and maintenance all done for you by us.
The easiest way to get started with Plausible Analytics is with [our official managed service in the cloud](https://plausible.io/#pricing). It takes 2 minutes to start counting your stats with a worldwide CDN, high availability, backups, security and maintenance all done for you by us.
In order to be compliant with the GDPR and the Schrems II ruling, all visitor data for our managed service in the cloud is exclusively processed on servers and cloud infrastructure owned and operated by European providers. Your website data never leaves the EU.
Our managed hosting can save a substantial amount of developer time and resources. For most sites this ends up being the best value option and the revenue goes to funding the maintenance and further development of Plausible. So you’ll be supporting open source software and getting a great service!
### Can Plausible Analytics be self-hosted?
### Can Plausible be self-hosted?
Plausible is [open source web analytics](https://plausible.io/open-source-website-analytics) and we have a free as in beer [self-hosted solution](https://plausible.io/self-hosted-web-analytics). Here are the differences between Plausible Analytics managed hosting in the cloud and the self-hosting:
Plausible is [open source web analytics](https://plausible.io/open-source-website-analytics) and we have a free as in beer and self-hosted solution called [Plausible Community Edition (CE)](https://plausible.io/self-hosted-web-analytics). Here are the differences between Plausible Analytics managed hosting in the cloud and the Plausible CE:
| | Plausible Cloud | Self-Hosting |
| | Plausible Analytics Cloud | Plausible Community Edition |
| ------------- | ------------- | ------------- |
| Hosting | Easy and convenient. It takes 2 minutes to start counting your stats with a worldwide CDN, high availability, backups, security and maintenance all done for you by us. We manage everything so you don’t have to worry about anything and can focus on your stats. | You do it all yourself. You need to get a server and you need to manage your infrastructure. You are responsible for installation, maintenance, upgrades, server capacity, uptime, backup, security, stability, consistency, loading time and so on. |
| Storage | All visitor data is exclusively processed on EU-owned cloud infrastructure. We keep your site data on a secure, encrypted and green energy powered server in Germany. This ensures that your site data is protected by the strict European Union data privacy laws and ensures compliance with GDPR. Your website data never leaves the EU. | You have full control and can host your instance on any server in any country that you wish. Host it on a server in your basement or host it with any cloud provider wherever you want, even those that are not GDPR compliant. |
| Features | Continuously developed and improved with new features and updates multiple times per week. | [It's a long term release](https://plausible.io/blog/building-open-source) published twice per year. Selected premium features are not available as we aim to ensure a protective barrier around our cloud offering. |
| Bot filtering | Advanced bot filtering for more accurate stats. Our algorithm detects and excludes non-human traffic patterns. We also exclude known bots by the User-Agent header and filter out traffic from known referrer spam domains. | Basic bot filtering that targets the most common non-human traffic based on the User-Agent header and referrer spam domains. |
| Raw data | You see all your site stats and metrics on our modern-looking, simple to use and fast loading dashboard. You can only see the stats aggregated in the dashboard. You can download the stats using the [CSV export](https://plausible.io/docs/export-stats), [stats API](https://plausible.io/docs/stats-api) or tools such as the [Data Studio Connector](https://plausible.io/docs/integration-guides#google-data-studio). | Do you want access to the raw data? Self-hosting gives you that option. You can take the data directly from the ClickHouse database. |
| Premium Support | Real support delivered by real human beings who build and maintain Plausible. | Premium support is not included. Self-hosting is community supported only. |
| Costs | There's a cost associated with providing an analytics service so we charge a subscription fee. We choose the subscription business model rather than the business model of surveillance capitalism. Your money funds further development of Plausible. | You need to pay for your server, CDN, backups and whatever other cost there is associated with running the infrastructure. You never have to pay any fees to us. Your money goes to 3rd party companies with no connection to us.|
| **Infrastructure management** | Easy and convenient. It takes 2 minutes to start counting your stats with a worldwide CDN, high availability, backups, security and maintenance all done for you by us. We manage everything so you don’t have to worry about anything and can focus on your stats. | You do it all yourself. You need to get a server and you need to manage your infrastructure. You are responsible for installation, maintenance, upgrades, server capacity, uptime, backup, security, stability, consistency, loading time and so on.|
| **Release schedule** | Continuously developed and improved with new features and updates multiple times per week. | [It's a long term release](https://plausible.io/blog/building-open-source) published twice per year so latest features and improvements won't be immediately available.|
| **Premium features** | All features available as listed in [our pricing plans](https://plausible.io/#pricing). | Selected premium features such as funnels and ecommerce revenue goals are not available as we aim to ensure a [protective barrier around our cloud offering](https://plausible.io/blog/community-edition).|
| **Bot filtering** | Advanced bot filtering for more accurate stats. Our algorithm detects and excludes non-human traffic patterns. We also exclude known bots by the User-Agent header and filter out traffic from data centers and referrer spam domains. | Basic bot filtering that targets the most common non-human traffic based on the User-Agent header and referrer spam domains.|
| **Server location** | All visitor data is exclusively processed on EU-owned cloud infrastructure. We keep your site data on a secure, encrypted and green energy powered server in Germany. This ensures that your site data is protected by the strict European Union data privacy laws and ensures compliance with GDPR. Your website data never leaves the EU. | You have full control and can host your instance on any server in any country that you wish. Host it on a server in your basement or host it with any cloud provider wherever you want, even those that are not GDPR compliant.|
| **Data portability** | You see all your site stats and metrics on our modern-looking, simple to use and fast loading dashboard. You can only see the stats aggregated in the dashboard. You can download the stats using the [CSV export](https://plausible.io/docs/export-stats), [stats API](https://plausible.io/docs/stats-api) or tools such as the [Data Studio Connector](https://plausible.io/docs/integration-guides#google-data-studio). | Do you want access to the raw data? Self-hosting gives you that option. You can take the data directly from the ClickHouse database. |
| **Premium support** | Real support delivered by real human beings who build and maintain Plausible. | Premium support is not included. CE is community supported only.|
| **Costs** | There's a cost associated with providing an analytics service so we charge a subscription fee. We choose the subscription business model rather than the business model of surveillance capitalism. Your money funds further development of Plausible. | You need to pay for your server, CDN, backups and whatever other cost there is associated with running the infrastructure. You never have to pay any fees to us. Your money goes to 3rd party companies with no connection to us.|
Interested in self-hosting Plausible on your server? Take a look at our [self-hosting installation instructions](https://plausible.io/docs/self-hosting).
Interested in self-hosting Plausible CE on your server? Take a look at our [Plausible CE installation instructions](https://github.com/plausible/community-edition/).
Plausible Self-Hosted is a community supported project and there are no guarantees that you will get support from the creators of Plausible to troubleshoot your self-hosting issues. There is a [community supported forum](https://github.com/plausible/analytics/discussions/categories/self-hosted-support) where you can ask for help.
Plausible CE is a community supported project and there are no guarantees that you will get support from the creators of Plausible to troubleshoot your self-hosting issues. There is a [community supported forum](https://github.com/plausible/analytics/discussions/categories/self-hosted-support) where you can ask for help.
Our only source of funding is our premium, managed service for running Plausible in the cloud. If you're looking for an alternative way to support the project, we've put together some sponsorship packages. If you choose to self-host Plausible you can [become a sponsor](https://github.com/sponsors/plausible) which is a great way to give back to the community and to contribute to the long-term sustainability of the project.
Our only source of funding is our premium, managed service for running Plausible in the cloud.
## Technology
@ -94,7 +95,7 @@ To stay up to date with all the latest news and product updates, make sure to fo
## License & Trademarks
Plausible is open source under the GNU Affero General Public License Version 3 (AGPLv3) or any later version. You can [find it here](https://github.com/plausible/analytics/blob/master/LICENSE.md).
Plausible CE is open source under the GNU Affero General Public License Version 3 (AGPLv3) or any later version. You can [find it here](https://github.com/plausible/analytics/blob/master/LICENSE.md).
To avoid issues with AGPL virality, we've released the JavaScript tracker which gets included on your website under the MIT license. You can [find it here](https://github.com/plausible/analytics/blob/master/tracker/LICENSE.md).

@ -7,6 +7,8 @@ want be up to date on security patches, make sure your Plausible image is up to
## Reporting a Vulnerability
Our software is updated several times per week and we also have a way for you to [report any security vulnerabilities](https://plausible.io/vulnerability-disclosure-program). A more detailed overview about our security practices can be found on [plausible.io/security](https://plausible.io/security).
If you've found a security vulnerability with the Plausible codebase, you can disclose it responsibly by sending a summary to security@plausible.io.
We will review the potential threat and fix it as fast as we can.

@ -9,6 +9,13 @@
[x-cloak] { display: none; }
@media print {
canvas {
width: 100% !important;
height: auto !important;
}
}
.button {
@apply inline-flex justify-center px-4 py-2 text-sm font-medium text-white bg-indigo-600 border border-transparent rounded-md leading-5 transition hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500;
}

@ -1,5 +1,4 @@
import "./polyfills/closest"
import "./apply-theme"
import 'abortcontroller-polyfill/dist/polyfill-patch-fetch'
import Alpine from 'alpinejs'
import "./liveview/live_socket"
@ -11,6 +10,16 @@ Alpine.data('dropdown', dropdown)
Alpine.data('comboBox', comboBox)
Alpine.start()
if (document.querySelectorAll('[data-modal]').length > 0) {
window.addEventListener(`phx:close-modal`, (e) => {
document
.getElementById(e.detail.id)
.dispatchEvent(
new CustomEvent('close-modal', { bubbles: true, detail: e.detail.id })
)
})
}
const triggers = document.querySelectorAll('[data-dropdown-trigger]')
for (const trigger of triggers) {

@ -1,37 +0,0 @@
var pref = document.currentScript.dataset.pref;
function reapplyTheme() {
var userPref = pref || "system";
var mediaPref = window.matchMedia('(prefers-color-scheme: dark)').matches;
var htmlRef = document.querySelector('html');
var hcaptchaRefs = document.getElementsByClassName('h-captcha');
htmlRef.classList.remove('dark');
for (let i = 0; i < hcaptchaRefs.length; i++) {
hcaptchaRefs[i].dataset.theme = "light";
}
switch (userPref) {
case "dark":
htmlRef.classList.add('dark');
for (let i = 0; i < hcaptchaRefs.length; i++) {
hcaptchaRefs[i].dataset.theme = "dark";
}
break;
case "system":
if (mediaPref) {
htmlRef.classList.add('dark');
for (let i = 0; i < hcaptchaRefs.length; i++) {
hcaptchaRefs[i].dataset.theme = "dark";
}
}
break;
}
}
reapplyTheme();
window.matchMedia('(prefers-color-scheme: dark)').addEventListener('change', reapplyTheme);
window.onload = function() {
reapplyTheme();
};

@ -16,10 +16,13 @@ if (container) {
domain: container.dataset.domain,
offset: container.dataset.offset,
hasGoals: container.dataset.hasGoals === 'true',
conversionsEnabled: container.dataset.conversionsEnabled === 'true',
funnelsEnabled: container.dataset.funnelsEnabled === 'true',
propsEnabled: container.dataset.propsEnabled === 'true',
hasProps: container.dataset.hasProps === 'true',
funnelsAvailable: container.dataset.funnelsAvailable === 'true',
propsAvailable: container.dataset.propsAvailable === 'true',
conversionsOptedOut: container.dataset.conversionsOptedOut === 'true',
funnelsOptedOut: container.dataset.funnelsOptedOut === 'true',
propsOptedOut: container.dataset.propsOptedOut === 'true',
revenueGoals: JSON.parse(container.dataset.revenueGoals),
funnels: JSON.parse(container.dataset.funnels),
statsBegin: container.dataset.statsBegin,
nativeStatsBegin: container.dataset.nativeStatsBegin,

@ -1,4 +1,4 @@
import {formatISO} from './util/date'
import { formatISO } from './util/date'
let abortController = new AbortController()
let SHARED_LINK_AUTH = null
@ -36,14 +36,15 @@ function serializeFilters(filters) {
return JSON.stringify(cleaned)
}
export function serializeQuery(query, extraQuery=[]) {
export function serializeQuery(query, extraQuery = []) {
const queryObj = {}
if (query.period) { queryObj.period = query.period }
if (query.date) { queryObj.date = formatISO(query.date) }
if (query.from) { queryObj.from = formatISO(query.from) }
if (query.to) { queryObj.to = formatISO(query.to) }
if (query.filters) { queryObj.filters = serializeFilters(query.filters) }
if (query.with_imported) { queryObj.with_imported = query.with_imported }
if (query.period) { queryObj.period = query.period }
if (query.date) { queryObj.date = formatISO(query.date) }
if (query.from) { queryObj.from = formatISO(query.from) }
if (query.to) { queryObj.to = formatISO(query.to) }
if (query.filters) { queryObj.filters = serializeFilters(query.filters) }
if (query.experimental_session_count) { queryObj.experimental_session_count = query.experimental_session_count }
if (query.with_imported) { queryObj.with_imported = query.with_imported }
if (SHARED_LINK_AUTH) { queryObj.auth = SHARED_LINK_AUTH }
if (query.comparison) {
@ -58,11 +59,11 @@ export function serializeQuery(query, extraQuery=[]) {
return '?' + serialize(queryObj)
}
export function get(url, query={}, ...extraQuery) {
const headers = SHARED_LINK_AUTH ? {'X-Shared-Link-Auth': SHARED_LINK_AUTH} : {}
export function get(url, query = {}, ...extraQuery) {
const headers = SHARED_LINK_AUTH ? { 'X-Shared-Link-Auth': SHARED_LINK_AUTH } : {}
url = url + serializeQuery(query, extraQuery)
return fetch(url, {signal: abortController.signal, headers: headers})
.then( response => {
return fetch(url, { signal: abortController.signal, headers: headers })
.then(response => {
if (!response.ok) {
return response.json().then((msg) => {
throw new ApiError(msg.error, msg)

@ -66,8 +66,9 @@ function ComparisonModeOption({ label, value, isCurrentlySelected, updateMode, s
}
const render = ({ active }) => {
const buttonClass = classNames("px-4 py-2 w-full text-left font-medium text-sm dark:text-white cursor-pointer", {
const buttonClass = classNames("px-4 py-2 w-full text-left text-sm dark:text-white", {
"bg-gray-100 text-gray-900 dark:bg-gray-900 dark:text-gray-100": active,
"font-medium": !isCurrentlySelected,
"font-bold": isCurrentlySelected,
})
@ -90,8 +91,9 @@ function MatchDayOfWeekInput({ history, query, site }) {
}
const buttonClass = (hover, selected) =>
classNames("px-4 py-2 w-full text-left font-medium text-sm dark:text-white cursor-pointer", {
classNames("px-4 py-2 w-full text-left text-sm dark:text-white cursor-pointer", {
"bg-gray-100 text-gray-900 dark:bg-gray-900 dark:text-gray-100": hover,
"font-medium": !selected,
"font-bold": selected,
})
@ -181,7 +183,7 @@ const ComparisonInput = function({ site, query, history }) {
</Transition>
{ uiMode == "datepicker" &&
<div className="h-0 absolute">
<div className="h-0 md:absolute">
<Flatpickr ref={calendar} options={flatpickrOptions} className="invisible" />
</div> }
</Menu>

@ -91,8 +91,10 @@ export default function PlausibleCombobox(props) {
}
}
function isDisabled(option) {
return props.values.some((val) => val.value === option.value)
function isOptionDisabled(option) {
const optionAlreadySelected = props.values.some((val) => val.value === option.value)
const optionDisabled = (props.disabledOptions || []).some((val) => val?.value === option.value)
return optionAlreadySelected || optionDisabled
}
function fetchOptions(query) {
@ -157,11 +159,11 @@ export default function PlausibleCombobox(props) {
}, [])
useEffect(() => {
if (props.singleOption && props.values.length === 0) {
if (props.singleOption && props.values.length === 0 && props.autoFocus) {
searchRef.current.focus()
}
}, [props.values.length === 0])
}, [props.values.length === 0, props.singleOption, props.autoFocus])
const searchBoxClass = 'border-none py-1 px-0 w-full inline-block rounded-md focus:outline-none focus:ring-0 text-sm'
const containerClass = classNames('relative w-full', {
@ -217,15 +219,15 @@ export default function PlausibleCombobox(props) {
}
function renderDropDownContent() {
const matchesFound = visibleOptions.length > 0 && visibleOptions.some(option => !isDisabled(option))
const matchesFound = visibleOptions.length > 0 && visibleOptions.some(option => !isOptionDisabled(option))
if (loading) {
return <div className="relative cursor-default select-none py-2 px-4 text-gray-700 dark:text-gray-300">Loading options...</div>
}
if (matchesFound) {
return visibleOptions
.filter(option => !isDisabled(option))
.filter(option => !isOptionDisabled(option))
.map((option, i) => {
const text = option.freeChoice ? `Filter by '${option.label}'` : option.label

@ -1,6 +1,6 @@
import React, { Fragment } from "react";
import { FILTER_TYPES } from "../util/filters";
import { FILTER_OPERATIONS } from "../util/filters";
import { Menu, Transition } from "@headlessui/react";
import { ChevronDownIcon } from '@heroicons/react/20/solid'
import { isFreeChoiceFilter, supportsIsNot } from "../util/filters";
@ -61,9 +61,9 @@ export default function FilterTypeSelector(props) {
className="z-10 origin-top-left absolute left-0 mt-2 w-full rounded-md shadow-lg bg-white dark:bg-gray-800 ring-1 ring-black ring-opacity-5 focus:outline-none"
>
<div className="py-1">
{renderTypeItem(FILTER_TYPES.is, true)}
{renderTypeItem(FILTER_TYPES.isNot, supportsIsNot(filterName))}
{renderTypeItem(FILTER_TYPES.contains, isFreeChoiceFilter(filterName))}
{renderTypeItem(FILTER_OPERATIONS.is, true)}
{renderTypeItem(FILTER_OPERATIONS.isNot, supportsIsNot(filterName))}
{renderTypeItem(FILTER_OPERATIONS.contains, isFreeChoiceFilter(filterName))}
</div>
</Menu.Items>
</Transition>

@ -2,7 +2,7 @@ import React from "react"
import { sectionTitles } from "../stats/behaviours"
import * as api from '../api'
export function FeatureSetupNotice({ site, feature, shortFeatureName, title, info, settingsLink, onHideAction }) {
export function FeatureSetupNotice({ site, feature, title, info, callToAction, onHideAction }) {
const sectionTitle = sectionTitles[feature]
const requestHideSection = () => {
@ -14,10 +14,10 @@ export function FeatureSetupNotice({ site, feature, shortFeatureName, title, inf
}
}
function setupButton() {
function renderCallToAction() {
return (
<a href={settingsLink} className="ml-2 sm:ml-4 button px-2 sm:px-4">
<p className="flex flex-col justify-center text-xs sm:text-sm">Set up {shortFeatureName}</p>
<a href={callToAction.link} className="ml-2 sm:ml-4 button px-2 sm:px-4">
<p className="flex flex-col justify-center text-xs sm:text-sm">{callToAction.action}</p>
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" strokeWidth={1.5} stroke="currentColor" className="ml-2 w-5 h-5">
<path strokeLinecap="round" strokeLinejoin="round" d="M4.5 12h15m0 0l-6.75-6.75M19.5 12l-6.75 6.75" />
</svg>
@ -25,7 +25,7 @@ export function FeatureSetupNotice({ site, feature, shortFeatureName, title, inf
)
}
function hideButton() {
function renderHideButton() {
return (
<button
onClick={requestHideSection}
@ -47,8 +47,8 @@ export function FeatureSetupNotice({ site, feature, shortFeatureName, title, inf
</div>
<div className="text-xs sm:text-sm flex my-6 justify-center">
{hideButton()}
{setupButton()}
{renderHideButton()}
{renderCallToAction()}
</div>
</div>
</div>

@ -0,0 +1,5 @@
Plausible copyright notice
This software is Copyright 2024 Plausible Insights OÜ
This repository is provided for informational purposes only, and no rights to use, distribute or otherwise exploit this software are granted to you. Our other repositories include software that is offered under the Affero GPL, but this repository is not. If you wish to discuss a license, or our hosted service that uses the software, please contact us at hello@plausible.io

@ -3,13 +3,13 @@ import FlipMove from 'react-flip-move';
import Chart from 'chart.js/auto'
import FunnelTooltip from './funnel-tooltip.js'
import ChartDataLabels from 'chartjs-plugin-datalabels'
import numberFormatter from '../../util/number-formatter'
import Bar from '../bar'
import numberFormatter from '../util/number-formatter'
import Bar from '../stats/bar'
import RocketIcon from '../modals/rocket-icon'
import RocketIcon from '../stats/modals/rocket-icon'
import * as api from '../../api'
import LazyLoader from '../../components/lazy-loader'
import * as api from '../api'
import LazyLoader from '../components/lazy-loader'
export default function Funnel(props) {

@ -7,4 +7,3 @@ export default function Money({ formatted }) {
return "-"
}
}

@ -10,12 +10,23 @@ import {
formatFilterGroup,
filterGroupForFilter,
parseQueryFilter,
parseQueryPropsFilter,
formattedFilters
} from "./util/filters";
function removeFilter(key, history, query) {
const newOpts = {
[key]: false
function removeFilter(filterType, key, history, query) {
const newOpts = {}
if (filterType === 'props') {
if (Object.keys(query.filters.props).length == 1) {
newOpts.props = false
} else {
newOpts.props = JSON.stringify({
...query.filters.props,
[key]: undefined,
})
}
} else {
newOpts[key] = false
}
if (key === 'country') { newOpts.country_labels = false }
if (key === 'region') { newOpts.region_labels = false }
@ -37,34 +48,38 @@ function clearAllFilters(history, query) {
);
}
function filterText(key, _rawValue, query) {
const {type, clauses} = parseQueryFilter(query, key)
function filterText(filterType, key, query) {
const formattedFilter = formattedFilters[key]
if (key === "props") {
const [[propKey, _propValue]] = Object.entries(query.filters['props'])
return <>Property <b>{propKey}</b> {type} {clauses.map(({label}) => <b key={label}>{label}</b>).reduce((prev, curr) => [prev, ' or ', curr])} </>
if (filterType === "props") {
const { propKey, clauses, type } = parseQueryPropsFilter(query).find((filter) => filter.propKey.value === key)
return <>Property <b>{propKey.label}</b> {type} {clauses.map(({label}) => <b key={label}>{label}</b>).reduce((prev, curr) => [prev, ' or ', curr])} </>
} else if (formattedFilter) {
const {type, clauses} = parseQueryFilter(query, key)
return <>{formattedFilter} {type} {clauses.map(({label}) => <b key={label}>{label}</b>).reduce((prev, curr) => [prev, ' or ', curr])} </>
}
throw new Error(`Unknown filter: ${key}`)
}
function renderDropdownFilter(site, history, [key, value], query) {
function renderDropdownFilter(site, history, { key, value, filterType }, query) {
return (
<Menu.Item key={key}>
<Menu.Item key={`${filterType}::${key}`}>
<div className="px-3 md:px-4 sm:py-2 py-3 text-sm leading-tight flex items-center justify-between" key={key + value}>
<Link
title={`Edit filter: ${formattedFilters[key]}`}
to={{ pathname: `/${encodeURIComponent(site.domain)}/filter/${filterGroupForFilter(key)}`, search: window.location.search }}
title={`Edit filter: ${formattedFilters[filterType]}`}
to={{ pathname: `/${encodeURIComponent(site.domain)}/filter/${filterGroupForFilter(filterType)}`, search: window.location.search }}
className="group flex w-full justify-between items-center"
style={{ width: 'calc(100% - 1.5rem)' }}
>
<span className="inline-block w-full truncate">{filterText(key, value, query)}</span>
<span className="inline-block w-full truncate">{filterText(filterType, key, query)}</span>
<PencilSquareIcon className="w-4 h-4 ml-1 cursor-pointer group-hover:text-indigo-700 dark:group-hover:text-indigo-500" />
</Link>
<b title={`Remove filter: ${formattedFilters[key]}`} className="ml-2 cursor-pointer hover:text-indigo-700 dark:hover:text-indigo-500" onClick={() => removeFilter(key, history, query)}>
<b
title={`Remove filter: ${formattedFilters[filterType]}`}
className="ml-2 cursor-pointer hover:text-indigo-700 dark:hover:text-indigo-500"
onClick={() => removeFilter(filterType, key, history, query)}
>
<XMarkIcon className="w-4 h-4" />
</b>
</div>
@ -95,7 +110,7 @@ function DropdownContent({ history, site, query, wrapped }) {
if (wrapped === 0 || addingFilter) {
let filterGroups = {...FILTER_GROUPS}
if (!site.propsEnabled) delete filterGroups.props
if (!site.propsAvailable) delete filterGroups.props
return Object.keys(filterGroups).map((option) => filterDropdownOption(site, option))
}
@ -202,13 +217,21 @@ class Filters extends React.Component {
});
};
renderListFilter(history, [key, value], query) {
renderListFilter(history, { key, value, filterType }, query) {
return (
<span key={key} title={value} className="flex bg-white dark:bg-gray-800 text-gray-700 dark:text-gray-300 shadow text-sm rounded mr-2 items-center">
<Link title={`Edit filter: ${formattedFilters[key]}`} className="flex w-full h-full items-center py-2 pl-3" to={{ pathname: `/${encodeURIComponent(this.props.site.domain)}/filter/${filterGroupForFilter(key)}`, search: window.location.search }}>
<span className="inline-block max-w-2xs md:max-w-xs truncate">{filterText(key, value, query)}</span>
<span key={`${filterType}::${key}`} title={value} className="flex bg-white dark:bg-gray-800 text-gray-700 dark:text-gray-300 shadow text-sm rounded mr-2 items-center">
<Link
title={`Edit filter: ${formattedFilters[filterType]}`}
className="flex w-full h-full items-center py-2 pl-3"
to={{ pathname: `/${encodeURIComponent(this.props.site.domain)}/filter/${filterGroupForFilter(filterType)}`, search: window.location.search }}
>
<span className="inline-block max-w-2xs md:max-w-xs truncate">{filterText(filterType, key, query)}</span>
</Link>
<span title={`Remove filter: ${formattedFilters[key]}`} className="flex h-full w-full px-2 cursor-pointer hover:text-indigo-700 dark:hover:text-indigo-500 items-center" onClick={() => removeFilter(key, history, query)}>
<span
title={`Remove filter: ${formattedFilters[filterType]}`}
className="flex h-full w-full px-2 cursor-pointer hover:text-indigo-700 dark:hover:text-indigo-500 items-center"
onClick={() => removeFilter(filterType, key, history, query)}
>
<XMarkIcon className="w-4 h-4" />
</span>
</span>
@ -235,6 +258,10 @@ class Filters extends React.Component {
)
}
trackFilterMenu() {
window.plausible && window.plausible('Filter Menu: Open', {u: `${window.location.protocol}//${window.location.hostname}/:dashboard`})
}
renderDropDown() {
const { history, query, site } = this.props;
@ -243,7 +270,7 @@ class Filters extends React.Component {
{({ open }) => (
<>
<div>
<Menu.Button className="flex items-center text-xs md:text-sm font-medium leading-tight px-3 py-2 cursor-pointer ml-auto text-gray-500 dark:text-gray-200 hover:bg-gray-200 dark:hover:bg-gray-900 rounded">
<Menu.Button onClick={this.trackFilterMenu} className="flex items-center text-xs md:text-sm font-medium leading-tight px-3 py-2 cursor-pointer ml-auto text-gray-500 dark:text-gray-200 hover:bg-gray-200 dark:hover:bg-gray-900 rounded">
{this.renderDropdownButton()}
</Menu.Button>
</div>

@ -1,8 +1,8 @@
import React from 'react'
import { Link, withRouter } from 'react-router-dom'
import {nowForSite} from './util/date'
import { nowForSite } from './util/date'
import * as storage from './util/storage'
import { COMPARISON_DISABLED_PERIODS, getStoredComparisonMode, isComparisonEnabled,getStoredMatchDayOfWeek } from './comparison-input'
import { COMPARISON_DISABLED_PERIODS, getStoredComparisonMode, isComparisonEnabled, getStoredMatchDayOfWeek } from './comparison-input'
import dayjs from 'dayjs';
import utc from 'dayjs/plugin/utc';
@ -39,6 +39,7 @@ export function parseQuery(querystring, site) {
to: q.get('to') ? dayjs.utc(q.get('to')) : undefined,
match_day_of_week: matchDayOfWeek == 'true',
with_imported: q.get('with_imported') ? q.get('with_imported') === 'true' : true,
experimental_session_count: q.get('experimental_session_count'),
filters: {
'goal': q.get('goal'),
'props': JSON.parse(q.get('props')),
@ -58,6 +59,7 @@ export function parseQuery(querystring, site) {
'region': q.get('region'),
'city': q.get('city'),
'page': q.get('page'),
'hostname': q.get('hostname'),
'entry_page': q.get('entry_page'),
'exit_page': q.get('exit_page')
}
@ -65,9 +67,13 @@ export function parseQuery(querystring, site) {
}
export function appliedFilters(query) {
return Object.keys(query.filters)
.map((key) => [key, query.filters[key]])
.filter(([_key, value]) => !!value);
const propKeys = Object.entries(query.filters.props || {})
.map(([key, value]) => ({ key, value, filterType: 'props' }))
return Object.entries(query.filters)
.map(([key, value]) => ({ key, value, filterType: key }))
.filter(({ key, value }) => key !== 'props' && !!value)
.concat(propKeys)
}
function generateQueryString(data) {
@ -122,7 +128,7 @@ class QueryLink extends React.Component {
const QueryLinkWithRouter = withRouter(QueryLink)
export { QueryLinkWithRouter as QueryLink };
function QueryButton({history, query, to, disabled, className, children, onClick}) {
function QueryButton({ history, query, to, disabled, className, children, onClick }) {
return (
<button
className={className}

@ -52,7 +52,7 @@ export default function Router({ site, loggedIn, currentUserRole }) {
<ExitPagesModal site={site} />
</Route>
<Route path="/:domain/countries">
<ModalTable title="Top countries" site={site} endpoint={url.apiPath(site, '/countries')} filter={{ country: 'code', country_labels: 'name' }} keyLabel="Country" renderIcon={renderCountryIcon} />
<ModalTable title="Top countries" site={site} endpoint={url.apiPath(site, '/countries')} filter={{ country: 'code', country_labels: 'name' }} keyLabel="Country" renderIcon={renderCountryIcon} showPercentage={true}/>
</Route>
<Route path="/:domain/regions">
<ModalTable title="Top regions" site={site} endpoint={url.apiPath(site, '/regions')} filter={{ region: 'code', region_labels: 'name' }} keyLabel="Region" renderIcon={renderRegionIcon} />

@ -164,7 +164,9 @@ export default class SiteSwitcher extends React.Component {
<React.Fragment>
<div className="py-1">
<a
href={`/${encodeURIComponent(this.props.site.domain)}/settings`}
href={`/${encodeURIComponent(
this.props.site.domain
)}/settings/general`}
className="group flex items-center px-4 py-2 md:text-sm leading-5 text-gray-700 dark:text-gray-300 hover:bg-gray-100 dark:hover:bg-gray-900 hover:text-gray-900 dark:hover:text-gray-100 focus:outline-none focus:bg-gray-100 dark:focus:bg-gray-900 focus:text-gray-900 dark:focus:text-gray-100"
role="menuitem"
>

@ -7,16 +7,17 @@ import { CR_METRIC } from '../reports/metrics';
import ListReport from '../reports/list';
export default function Conversions(props) {
const {site, query} = props
const { site, query } = props
function fetchConversions() {
return api.get(url.apiPath(site, '/conversions'), query, {limit: 9})
return api.get(url.apiPath(site, '/conversions'), query, { limit: 9 })
}
function getFilterFor(listItem) {
return {goal: escapeFilterValue(listItem.name)}
return { goal: escapeFilterValue(listItem.name) }
}
/*global BUILD_EXTRA*/
return (
<ListReport
fetchData={fetchConversions}
@ -24,11 +25,11 @@ export default function Conversions(props) {
keyLabel="Goal"
onClick={props.onGoalFilterClick}
metrics={[
{name: 'visitors', label: "Uniques", plot: true},
{name: 'events', label: "Total", hiddenOnMobile: true},
{ name: 'visitors', label: "Uniques", plot: true },
{ name: 'events', label: "Total", hiddenOnMobile: true },
CR_METRIC,
{name: 'total_revenue', label: 'Revenue', hiddenOnMobile: true},
{name: 'average_revenue', label: 'Average', hiddenOnMobile: true}
BUILD_EXTRA && { name: 'total_revenue', label: 'Revenue', hiddenOnMobile: true },
BUILD_EXTRA && { name: 'average_revenue', label: 'Average', hiddenOnMobile: true }
]}
detailsLink={url.sitePath(site, '/conversions')}
maybeHideDetails={true}

@ -6,10 +6,21 @@ import * as storage from '../../util/storage'
import GoalConversions, { specialTitleWhenGoalFilter } from './goal-conversions'
import Properties from './props'
import Funnel from './funnel'
import { FeatureSetupNotice } from '../../components/notice'
import { SPECIAL_GOALS } from './goal-conversions'
/*global BUILD_EXTRA*/
/*global require*/
function maybeRequire() {
if (BUILD_EXTRA) {
return require('../../extra/funnel')
} else {
return { default: null }
}
}
const Funnel = maybeRequire().default
const ACTIVE_CLASS = 'inline-block h-5 text-indigo-700 dark:text-indigo-500 font-bold active-prop-heading truncate text-left'
const DEFAULT_CLASS = 'hover:text-indigo-600 cursor-pointer truncate text-left'
@ -32,7 +43,7 @@ export default function Behaviours(props) {
const [mode, setMode] = useState(defaultMode())
const [funnelNames, _setFunnelNames] = useState(site.funnels.map(({ name }) => name))
const [selectedFunnel, setSelectedFunnel] = useState(storage.getItem(funnelKey))
const [selectedFunnel, setSelectedFunnel] = useState(defaultSelectedFunnel())
const [showingPropsForGoalFilter, setShowingPropsForGoalFilter] = useState(false)
@ -72,8 +83,22 @@ export default function Behaviours(props) {
}
}
function defaultSelectedFunnel() {
const stored = storage.getItem(funnelKey)
const storedExists = stored && site.funnels.some((f) => f.name === stored)
if (storedExists) {
return stored
} else if (site.funnels.length > 0) {
const firstAvailable = site.funnels[0].name
storage.setItem(funnelKey, firstAvailable)
return firstAvailable
}
}
function hasFunnels() {
return site.funnels.length > 0
return site.funnels.length > 0 && site.funnelsAvailable
}
function tabFunnelPicker() {
@ -139,7 +164,7 @@ export default function Behaviours(props) {
<div className="flex text-xs font-medium text-gray-500 dark:text-gray-400 space-x-2">
{isEnabled(CONVERSIONS) && tabSwitcher(CONVERSIONS, 'Goals')}
{isEnabled(PROPS) && tabSwitcher(PROPS, 'Properties')}
{isEnabled(FUNNELS) && (hasFunnels() ? tabFunnelPicker() : tabSwitcher(FUNNELS, 'Funnels'))}
{isEnabled(FUNNELS) && Funnel && (hasFunnels() ? tabFunnelPicker() : tabSwitcher(FUNNELS, 'Funnels'))}
</div>
)
}
@ -153,10 +178,12 @@ export default function Behaviours(props) {
<FeatureSetupNotice
site={site}
feature={CONVERSIONS}
shortFeatureName={'goals'}
title={'Measure how often visitors complete specific actions'}
info={'Goals allow you to track registrations, button clicks, form completions, external link clicks, file downloads, 404 error pages and more.'}
settingsLink={`/${encodeURIComponent(site.domain)}/settings/goals`}
callToAction={{
action: "Set up goals",
link: `/${encodeURIComponent(site.domain)}/settings/goals`
}}
onHideAction={onHideAction(CONVERSIONS)}
/>
)
@ -165,16 +192,28 @@ export default function Behaviours(props) {
}
function renderFunnels() {
if (selectedFunnel) { return <Funnel site={site} query={query} funnelName={selectedFunnel} /> }
else if (adminAccess) {
if (Funnel === null) {
return featureUnavailable()
}
else if (Funnel && selectedFunnel && site.funnelsAvailable) {
return <Funnel site={site} query={query} funnelName={selectedFunnel} />
}
else if (Funnel && adminAccess) {
let callToAction
if (site.funnelsAvailable) {
callToAction = {action: 'Set up funnels', link: `/${encodeURIComponent(site.domain)}/settings/funnels`}
} else {
callToAction = {action: 'Upgrade', link: '/billing/choose-plan'}
}
return (
<FeatureSetupNotice
site={site}
feature={FUNNELS}
shortFeatureName={'funnels'}
title={'Follow the visitor journey from entry to conversion'}
info={'Funnels allow you to analyze the user flow through your website, uncover possible issues, optimize your site and increase the conversion rate.'}
settingsLink={`/${encodeURIComponent(site.domain)}/settings/funnels`}
callToAction={callToAction}
onHideAction={onHideAction(FUNNELS)}
/>
)
@ -183,17 +222,24 @@ export default function Behaviours(props) {
}
function renderProps() {
if (site.hasProps) {
if (site.hasProps && site.propsAvailable) {
return <Properties site={site} query={query} />
} else if (adminAccess) {
let callToAction
if (site.propsAvailable) {
callToAction = {action: 'Set up props', link: `/${encodeURIComponent(site.domain)}/settings/properties`}
} else {
callToAction = {action: 'Upgrade', link: '/billing/choose-plan'}
}
return (
<FeatureSetupNotice
site={site}
feature={PROPS}
shortFeatureName={'props'}
title={'No custom properties found'}
title={'Send custom data to create your own metrics'}
info={'You can attach custom properties when sending a pageview or event. This allows you to create custom metrics and analyze stats we don\'t track automatically.'}
settingsLink={`/${encodeURIComponent(site.domain)}/settings/properties`}
callToAction={callToAction}
onHideAction={onHideAction(PROPS)}
/>
)
@ -208,6 +254,14 @@ export default function Behaviours(props) {
)
}
function featureUnavailable() {
return (
<div className="font-medium text-gray-500 dark:text-gray-400 py-12 text-center">
This feature is unavailable
</div>
)
}
function onHideAction(mode) {
return () => { disableMode(mode) }
}
@ -237,15 +291,20 @@ export default function Behaviours(props) {
function getEnabledModes() {
let enabledModes = []
if (site.conversionsEnabled) {
enabledModes.push(CONVERSIONS)
}
if (site.propsEnabled) {
enabledModes.push(PROPS)
}
if (site.funnelsEnabled && !isRealtime()) {
enabledModes.push(FUNNELS)
for (const feature of Object.keys(sectionTitles)) {
const isOptedOut = site[feature + 'OptedOut']
const isAvailable = site[feature + 'Available'] !== false
// If the feature is not supported by the site owner's subscription,
// it only makes sense to display the feature tab to the owner itself
// as only they can upgrade to make the feature available.
const callToActionIsMissing = !isAvailable && currentUserRole !== 'owner'
if (!isOptedOut && !callToActionIsMissing) {
enabledModes.push(feature)
}
}
return enabledModes
}

@ -1,199 +0,0 @@
import React from 'react';
import { Link } from 'react-router-dom'
import * as storage from '../../util/storage'
import Bar from '../bar'
import numberFormatter from '../../util/number-formatter'
import * as api from '../../api'
import Money from './money'
import { isValidHttpUrl, trimURL } from '../../util/url'
const MOBILE_UPPER_WIDTH = 767
const DEFAULT_WIDTH = 1080
const BREAKDOWN_LIMIT = 100
export default class PropertyBreakdown extends React.Component {
constructor(props) {
super(props)
let propKey = props.goal.prop_names[0]
this.storageKey = 'goalPropTab__' + props.site.domain + props.goal.name
const storedKey = storage.getItem(this.storageKey)
if (props.goal.prop_names.includes(storedKey)) {
propKey = storedKey
}
if (props.query.filters['props']) {
propKey = Object.keys(props.query.filters['props'])[0]
}
this.state = {
loading: true,
propKey: propKey,
viewport: DEFAULT_WIDTH,
breakdown: [],
page: 1,
moreResultsAvailable: false
}
this.handleResize = this.handleResize.bind(this);
this.fetch = this.fetch.bind(this)
this.fetchAndReplace = this.fetchAndReplace.bind(this)
this.fetchAndConcat = this.fetchAndConcat.bind(this)
}
componentDidMount() {
window.addEventListener('resize', this.handleResize, false);
this.handleResize();
this.fetchAndReplace()
if (this.props.query.period === 'realtime') {
document.addEventListener('tick', this.fetchAndReplace)
}
}
componentWillUnmount() {
window.removeEventListener('resize', this.handleResize, false);
document.removeEventListener('tick', this.fetchAndReplace)
}
handleResize() {
this.setState({ viewport: window.innerWidth });
}
fetch({ concat }) {
if (!this.props.query.filters['goal']) return
api.get(`/api/stats/${encodeURIComponent(this.props.site.domain)}/property/${encodeURIComponent(this.state.propKey)}`, this.props.query, { limit: BREAKDOWN_LIMIT, page: this.state.page })
.then((res) => {
let breakdown = concat ? this.state.breakdown.concat(res) : res
this.setState(() => ({
loading: false,
breakdown: breakdown,
moreResultsAvailable: res.length >= BREAKDOWN_LIMIT
}))
})
}
fetchAndReplace() {
this.fetch({ concat: false })
}
fetchAndConcat() {
this.fetch({ concat: true })
}
loadMore() {
this.setState({ loading: true, page: this.state.page + 1 }, this.fetchAndConcat.bind(this))
}
renderUrl(value) {
if (isValidHttpUrl(value.name)) {
return (
<a target="_blank" href={value.name} rel="noreferrer" className="hidden group-hover:block">
<svg className="inline h-4 w-4 ml-1 -mt-1 text-gray-600 dark:text-gray-400" fill="currentColor" viewBox="0 0 20 20"><path d="M11 3a1 1 0 100 2h2.586l-6.293 6.293a1 1 0 101.414 1.414L15 6.414V9a1 1 0 102 0V4a1 1 0 00-1-1h-5z"></path><path d="M5 5a2 2 0 00-2 2v8a2 2 0 002 2h8a2 2 0 002-2v-3a1 1 0 10-2 0v3H5V7h3a1 1 0 000-2H5z"></path></svg>
</a>
)
}
return null
}
renderPropContent(value, query) {
return (
<span className="flex px-2 py-1.5 group dark:text-gray-300 relative z-9 break-all">
<Link
to={{ pathname: window.location.pathname, search: query.toString() }}
className="md:truncate hover:underline block"
>
{trimURL(value.name, 100)}
</Link>
{this.renderUrl(value)}
</span>
)
}
renderPropValue(value) {
const query = new URLSearchParams(window.location.search)
query.set('props', JSON.stringify({ [this.state.propKey]: value.name }))
const { viewport } = this.state;
return (
<div className="flex items-center justify-between my-2" key={value.name}>
<div className="flex-1 truncate">
<Bar
count={value.unique_conversions}
plot="unique_conversions"
all={this.state.breakdown}
bg="bg-red-50 dark:bg-gray-500 dark:bg-opacity-15"
>
{this.renderPropContent(value, query)}
</Bar>
</div>
<div className="flex dark:text-gray-200">
<span className="font-medium inline-block w-20 text-right">{numberFormatter(value.unique_conversions)}</span>
{
viewport > MOBILE_UPPER_WIDTH ?
(
<span
className="font-medium inline-block w-20 text-right"
>{numberFormatter(value.total_conversions)}
</span>
)
: null
}
<span className="font-medium inline-block w-20 text-right">{numberFormatter(value.conversion_rate)}%</span>
{this.props.renderRevenueColumn && <span className="hidden md:inline-block md:w-20 font-medium text-right"><Money formatted={value.total_revenue} /></span>}
{this.props.renderRevenueColumn && <span className="hidden md:inline-block md:w-20 font-medium text-right"><Money formatted={value.average_revenue} /></span>}
</div>
</div>
)
}
changePropKey(newKey) {
storage.setItem(this.storageKey, newKey)
this.setState({ propKey: newKey, loading: true, breakdown: [], page: 1, moreResultsAvailable: false }, this.fetchAndReplace)
}
renderLoading() {
if (this.state.loading) {
return <div className="px-4 py-2"><div className="loading sm mx-auto"><div></div></div></div>
} else if (this.state.moreResultsAvailable && this.props.query.period !== 'realtime') {
return (
<div className="w-full text-center my-4">
<button onClick={this.loadMore.bind(this)} type="button" className="button">
Load more
</button>
</div>
)
}
}
renderBody() {
return this.state.breakdown.map((propValue) => this.renderPropValue(propValue))
}
renderPill(key) {
const isActive = this.state.propKey === key
if (isActive) {
return <li key={key} className="inline-block h-5 text-indigo-700 dark:text-indigo-500 font-bold mr-2 active-prop-heading">{key}</li>
} else {
return <li key={key} className="hover:text-indigo-600 cursor-pointer mr-2" onClick={this.changePropKey.bind(this, key)}>{key}</li>
}
}
render() {
return (
<div className="w-full pl-3 sm:pl-6 mt-4">
<div className="flex-col sm:flex-row flex items-center pb-1">
<span className="text-xs font-bold text-gray-600 dark:text-gray-300 self-start sm:self-auto mb-1 sm:mb-0">Breakdown by:</span>
<ul className="flex flex-wrap font-medium text-xs text-gray-500 dark:text-gray-400 leading-5 pl-1 sm:pl-2">
{this.props.goal.prop_names.map(this.renderPill.bind(this))}
</ul>
</div>
{this.renderBody()}
{this.renderLoading()}
</div>
)
}
}

@ -1,4 +1,4 @@
import React, { useCallback, useState, useEffect } from "react";
import React, { useCallback, useState } from "react"
import ListReport from "../reports/list";
import Combobox from '../../components/combobox'
import * as api from '../../api'
@ -12,12 +12,8 @@ export default function Properties(props) {
const { site, query } = props
const propKeyStorageName = `prop_key__${site.domain}`
const propKeyStorageNameForGoal = `${query.filters.goal}__prop_key__${site.domain}`
const [propKey, setPropKey] = useState(choosePropKey())
useEffect(() => {
setPropKey(choosePropKey())
}, [query.filters.goal, query.filters.props])
const [propKey, setPropKey] = useState(choosePropKey())
function singleGoalFilterApplied() {
const goalFilter = query.filters.goal
@ -42,7 +38,7 @@ export default function Properties(props) {
const storedForGoal = storage.getItem(propKeyStorageNameForGoal)
if (storedForGoal) { return storedForGoal }
}
return storage.getItem(propKeyStorageName)
}
@ -69,6 +65,7 @@ export default function Properties(props) {
}
}
/*global BUILD_EXTRA*/
function renderBreakdown() {
return (
<ListReport
@ -76,11 +73,11 @@ export default function Properties(props) {
getFilterFor={getFilterFor}
keyLabel={propKey}
metrics={[
{name: 'visitors', label: 'Visitors', plot: true},
{name: 'events', label: 'Events', hiddenOnMobile: true},
{ name: 'visitors', label: 'Visitors', plot: true },
{ name: 'events', label: 'Events', hiddenOnMobile: true },
query.filters.goal ? CR_METRIC : PERCENTAGE_METRIC,
{name: 'total_revenue', label: 'Revenue', hiddenOnMobile: true},
{name: 'average_revenue', label: 'Average', hiddenOnMobile: true}
BUILD_EXTRA && { name: 'total_revenue', label: 'Revenue', hiddenOnMobile: true },
BUILD_EXTRA && { name: 'average_revenue', label: 'Average', hiddenOnMobile: true }
]}
detailsLink={url.sitePath(site, `/custom-prop-values/${propKey}`)}
maybeHideDetails={true}
@ -91,16 +88,18 @@ export default function Properties(props) {
)
}
const getFilterFor = (listItem) => { return {'props': JSON.stringify({[propKey]: escapeFilterValue(listItem.name)})} }
const comboboxValues = propKey ? [{value: propKey, label: propKey}] : []
const getFilterFor = (listItem) => ({
props: JSON.stringify({ ...query.filters.props, [propKey]: escapeFilterValue(listItem.name) })
})
const comboboxValues = propKey ? [{ value: propKey, label: propKey }] : []
const boxClass = 'pl-2 pr-8 py-1 bg-transparent dark:text-gray-300 rounded-md shadow-sm border border-gray-300 dark:border-gray-500'
return (
<div className="w-full mt-4">
<div>
<Combobox isDisabled={!!query.filters.props} boxClass={boxClass} fetchOptions={fetchPropKeyOptions()} singleOption={true} values={comboboxValues} onSelect={onPropKeySelect()} placeholder={'Select a property'} />
</div>
{ propKey && renderBreakdown() }
<div>
<Combobox boxClass={boxClass} fetchOptions={fetchPropKeyOptions()} singleOption={true} values={comboboxValues} onSelect={onPropKeySelect()} placeholder={'Select a property'} />
</div>
{propKey && renderBreakdown()}
</div>
)
}
}

@ -1,6 +1,7 @@
import React from 'react';
import * as storage from '../../util/storage'
import { isFilteringOnFixedValue } from '../../util/filters'
import ListReport from '../reports/list'
import * as api from '../../api'
import * as url from '../../util/url'
@ -12,7 +13,7 @@ function Browsers({ query, site }) {
}
function getFilterFor(listItem) {
return { browser: listItem['name']}
return { browser: listItem['name'] }
}
return (
@ -35,7 +36,7 @@ function BrowserVersions({ query, site }) {
if (query.filters.browser === '(not set)') {
return {}
}
return { browser_version: listItem['name']}
return { browser_version: listItem['name'] }
}
return (
@ -56,7 +57,7 @@ function OperatingSystems({ query, site }) {
}
function getFilterFor(listItem) {
return { os: listItem['name']}
return { os: listItem['name'] }
}
return (
@ -79,7 +80,7 @@ function OperatingSystemVersions({ query, site }) {
if (query.filters.os === '(not set)') {
return {}
}
return { os_version: listItem['name']}
return { os_version: listItem['name'] }
}
return (
@ -104,7 +105,7 @@ function ScreenSizes({ query, site }) {
}
function getFilterFor(listItem) {
return { screen: listItem['name']}
return { screen: listItem['name'] }
}
return (
@ -161,12 +162,12 @@ export default class Devices extends React.Component {
renderContent() {
switch (this.state.mode) {
case 'browser':
if (this.props.query.filters.browser) {
if (isFilteringOnFixedValue(this.props.query, 'browser')) {
return <BrowserVersions site={this.props.site} query={this.props.query} />
}
return <Browsers site={this.props.site} query={this.props.query} />
case 'os':
if (this.props.query.filters.os) {
if (isFilteringOnFixedValue(this.props.query, 'os')) {
return <OperatingSystemVersions site={this.props.site} query={this.props.query} />
}
return <OperatingSystems site={this.props.site} query={this.props.query} />

@ -1,27 +1,53 @@
import numberFormatter, {durationFormatter} from '../../util/number-formatter'
import { parsePrefix } from '../../util/filters'
export const METRIC_MAPPING = {
'Unique visitors (last 30 min)': 'visitors',
'Pageviews (last 30 min)': 'pageviews',
'Unique visitors': 'visitors',
'Visit duration': 'visit_duration',
'Total pageviews': 'pageviews',
'Views per visit': 'views_per_visit',
'Total visits': 'visits',
'Bounce rate': 'bounce_rate',
'Unique conversions': 'conversions',
'Average revenue': 'average_revenue',
'Total revenue': 'total_revenue',
export function getGraphableMetrics(query, site) {
const isRealtime = query.period === 'realtime'
const goalFilter = query.filters.goal
const pageFilter = query.filters.page
if (isRealtime && goalFilter) {
return ["visitors"]
} else if (isRealtime) {
return ["visitors", "pageviews"]
} else if (goalFilter && canGraphRevenueMetrics(goalFilter, site)) {
return ["visitors", "events", "average_revenue", "total_revenue", "conversion_rate"]
} else if (goalFilter) {
return ["visitors", "events", "conversion_rate"]
} else if (pageFilter) {
return ["visitors", "visits", "pageviews", "bounce_rate", "time_on_page"]
} else {
return ["visitors", "visits", "pageviews", "views_per_visit", "bounce_rate", "visit_duration"]
}
}
// Revenue metrics can only be graphed if:
// * The query is filtered by at least one revenue goal
// * All revenue goals in filter have the same currency
function canGraphRevenueMetrics(goalFilter, site) {
const goalsInFilter = parsePrefix(goalFilter).values
const revenueGoalsInFilter = site.revenueGoals.filter((rg) => {
return goalsInFilter.includes(rg.event_name)
})
const singleCurrency = revenueGoalsInFilter.every((rg) => {
return rg.currency === revenueGoalsInFilter[0].currency
})
return revenueGoalsInFilter.length > 0 && singleCurrency
}
export const METRIC_LABELS = {
'visitors': 'Visitors',
'pageviews': 'Pageviews',
'events': 'Total Conversions',
'views_per_visit': 'Views per Visit',
'visits': 'Visits',
'bounce_rate': 'Bounce Rate',
'visit_duration': 'Visit Duration',
'conversions': 'Converted Visitors',
'conversion_rate': 'Conversion Rate',
'average_revenue': 'Average Revenue',
'total_revenue': 'Total Revenue',
}
@ -29,23 +55,17 @@ export const METRIC_LABELS = {
export const METRIC_FORMATTER = {
'visitors': numberFormatter,
'pageviews': numberFormatter,
'events': numberFormatter,
'visits': numberFormatter,
'views_per_visit': (number) => (number),
'bounce_rate': (number) => (`${number}%`),
'visit_duration': durationFormatter,
'conversions': numberFormatter,
'conversion_rate': (number) => (`${number}%`),
'total_revenue': numberFormatter,
'average_revenue': numberFormatter,
}
export const LoadingState = {
loading: 'loading',
refreshing: 'refreshing',
loaded: 'loaded',
isLoadingOrRefreshing: function (state) { return [this.loading, this.refreshing].includes(state) },
isLoadedOrRefreshing: function (state) { return [this.loaded, this.refreshing].includes(state) }
}
const buildComparisonDataset = function(comparisonPlot) {
if (!comparisonPlot) return []

@ -6,7 +6,7 @@ import * as storage from '../../util/storage'
import { isKeyPressed } from '../../keybinding.js'
import { monthsBetweenDates } from '../../util/date.js'
export const INTERVAL_LABELS = {
const INTERVAL_LABELS = {
'minute': 'Minutes',
'hour': 'Hours',
'date': 'Days',
@ -14,11 +14,19 @@ export const INTERVAL_LABELS = {
'month': 'Months'
}
export const getStoredInterval = function(period, domain) {
function validIntervals(site, query) {
if (query.period === "custom" && monthsBetweenDates(query.from, query.to) > 12) {
return ["week", "month"]
} else {
return site.validIntervalsByPeriod[query.period]
}
}
function getStoredInterval(period, domain) {
return storage.getItem(`interval__${period}__${domain}`)
}
export const storeInterval = function(period, domain, interval) {
function storeInterval(period, domain, interval) {
storage.setItem(`interval__${period}__${domain}`, interval)
}
@ -33,37 +41,47 @@ function subscribeKeybinding(element) {
}, [handleKeyPress])
}
function DropdownItem({ option, currentInterval, updateInterval }) {
return (
<Menu.Item onClick={() => updateInterval(option)} key={option} disabled={option == currentInterval}>
{({ active }) => (
<span className={classNames({
'bg-gray-100 dark:bg-gray-900 text-gray-900 dark:text-gray-200 cursor-pointer': active,
'text-gray-700 dark:text-gray-200': !active,
'font-bold cursor-none select-none': option == currentInterval,
}, 'block px-4 py-2 text-sm')}>
{INTERVAL_LABELS[option]}
</span>
)}
</Menu.Item>
)
export const getCurrentInterval = function(site, query) {
const options = validIntervals(site, query)
const storedInterval = getStoredInterval(query.period, site.domain)
const defaultInterval = [...options].pop()
if (storedInterval && options.includes(storedInterval)) {
return storedInterval
} else {
return defaultInterval
}
}
export function IntervalPicker({ graphData, query, site, updateInterval }) {
export function IntervalPicker({ query, site, onIntervalUpdate }) {
if (query.period == 'realtime') return null
const menuElement = React.useRef(null)
subscribeKeybinding(menuElement)
const options = validIntervals(site, query)
const currentInterval = getCurrentInterval(site, query)
let currentInterval = graphData?.interval
subscribeKeybinding(menuElement)
let options = site.validIntervalsByPeriod[query.period]
if (query.period === "custom" && monthsBetweenDates(query.from, query.to) > 12) {
options = ["week", "month"]
function updateInterval(interval) {
storeInterval(query.period, site.domain, interval)
onIntervalUpdate(interval)
}
if (!options.includes(currentInterval)) {
currentInterval = [...options].pop()
function renderDropdownItem(option) {
return (
<Menu.Item onClick={() => updateInterval(option)} key={option} disabled={option == currentInterval}>
{({ active }) => (
<span className={classNames({
'bg-gray-100 dark:bg-gray-900 text-gray-900 dark:text-gray-200 cursor-pointer': active,
'text-gray-700 dark:text-gray-200': !active,
'font-bold cursor-none select-none': option == currentInterval,
}, 'block px-4 py-2 text-sm')}>
{INTERVAL_LABELS[option]}
</span>
)}
</Menu.Item>
)
}
return (
@ -85,7 +103,7 @@ export function IntervalPicker({ graphData, query, site, updateInterval }) {
leaveFrom="opacity-100 scale-100"
leaveTo="opacity-0 scale-95">
<Menu.Items className="py-1 text-left origin-top-right absolute right-0 mt-2 w-56 rounded-md shadow-lg bg-white dark:bg-gray-800 ring-1 ring-black ring-opacity-5 focus:outline-none z-10" static>
{options.map((option) => DropdownItem({ option, currentInterval, updateInterval }))}
{options.map(renderDropdownItem)}
</Menu.Items>
</Transition>
</>

@ -0,0 +1,247 @@
import React from 'react';
import { withRouter } from 'react-router-dom'
import Chart from 'chart.js/auto';
import { navigateToQuery } from '../../query'
import GraphTooltip from './graph-tooltip'
import { buildDataSet, METRIC_LABELS, METRIC_FORMATTER } from './graph-util'
import dateFormatter from './date-formatter';
import FadeIn from '../../fade-in';
import classNames from 'classnames';
const calculateMaximumY = function(dataset) {
const yAxisValues = dataset
.flatMap((item) => item.data)
.map((item) => item || 0)
if (yAxisValues) {
return Math.max(...yAxisValues)
} else {
return 1
}
}
class LineGraph extends React.Component {
constructor(props) {
super(props);
this.regenerateChart = this.regenerateChart.bind(this);
this.updateWindowDimensions = this.updateWindowDimensions.bind(this);
}
getGraphMetric() {
let metric = this.props.graphData.metric
if (metric == 'visitors' && this.props.query.filters.goal) {
return 'conversions'
} else {
return metric
}
}
regenerateChart() {
const { graphData, query } = this.props
const metric = this.getGraphMetric()
const graphEl = document.getElementById("main-graph-canvas")
this.ctx = graphEl.getContext('2d');
const dataSet = buildDataSet(graphData.plot, graphData.comparison_plot, graphData.present_index, this.ctx, METRIC_LABELS[metric])
return new Chart(this.ctx, {
type: 'line',
data: {
labels: graphData.labels,
datasets: dataSet
},
options: {
animation: false,
plugins: {
legend: { display: false },
tooltip: {
enabled: false,
mode: 'index',
intersect: false,
position: 'average',
external: GraphTooltip(graphData, metric, query)
},
},
responsive: true,
maintainAspectRatio: false,
onResize: this.updateWindowDimensions,
elements: { line: { tension: 0 }, point: { radius: 0 } },
onClick: this.maybeHopToHoveredPeriod.bind(this),
scale: {
ticks: { precision: 0, maxTicksLimit: 8 }
},
scales: {
y: {
min: 0,
suggestedMax: calculateMaximumY(dataSet),
ticks: {
callback: METRIC_FORMATTER[metric],
color: this.props.darkTheme ? 'rgb(243, 244, 246)' : undefined
},
grid: {
zeroLineColor: 'transparent',
drawBorder: false,
}
},
yComparison: {
min: 0,
suggestedMax: calculateMaximumY(dataSet),
display: false,
grid: { display: false },
},
x: {
grid: { display: false },
ticks: {
callback: function(val, _index, _ticks) {
if (this.getLabelForValue(val) == "__blank__") return ""
const hasMultipleYears =
graphData.labels
.filter((date) => typeof date === 'string')
.map(date => date.split('-')[0])
.filter((value, index, list) => list.indexOf(value) === index)
.length > 1
if (graphData.interval === 'hour' && query.period !== 'day') {
const date = dateFormatter({
interval: "date",
longForm: false,
period: query.period,
shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
const hour = dateFormatter({
interval: graphData.interval,
longForm: false,
period: query.period,
shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
// Returns a combination of date and hour. This is because
// small intervals like hour may return multiple days
// depending on the query period.
return `${date}, ${hour}`
}
if (graphData.interval === 'minute' && query.period !== 'realtime') {
return dateFormatter({
interval: "hour", longForm: false, period: query.period,
})(this.getLabelForValue(val))
}
return dateFormatter({
interval: graphData.interval, longForm: false, period: query.period, shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
},
color: this.props.darkTheme ? 'rgb(243, 244, 246)' : undefined
}
}
},
interaction: {
mode: 'index',
intersect: false,
}
}
});
}
repositionTooltip(e) {
const tooltipEl = document.getElementById('chartjs-tooltip');
if (tooltipEl && window.innerWidth >= 768) {
if (e.clientX > 0.66 * window.innerWidth) {
tooltipEl.style.right = (window.innerWidth - e.clientX) + window.pageXOffset + 'px'
tooltipEl.style.left = null;
} else {
tooltipEl.style.right = null;
tooltipEl.style.left = e.clientX + window.pageXOffset + 'px'
}
tooltipEl.style.top = e.clientY + window.pageYOffset + 'px'
tooltipEl.style.opacity = 1;
}
}
componentDidMount() {
if (this.props.graphData) {
this.chart = this.regenerateChart();
}
window.addEventListener('mousemove', this.repositionTooltip);
}
componentDidUpdate(prevProps) {
const { graphData, darkTheme } = this.props;
const tooltip = document.getElementById('chartjs-tooltip');
if (
graphData !== prevProps.graphData ||
darkTheme !== prevProps.darkTheme
) {
if (graphData) {
if (this.chart) {
this.chart.destroy();
}
this.chart = this.regenerateChart();
this.chart.update();
}
if (tooltip) {
tooltip.style.display = 'none';
}
}
if (!graphData) {
if (this.chart) {
this.chart.destroy();
}
if (tooltip) {
tooltip.style.display = 'none';
}
}
}
componentWillUnmount() {
// Ensure that the tooltip doesn't hang around when we are loading more data
const tooltip = document.getElementById('chartjs-tooltip');
if (tooltip) {
tooltip.style.opacity = 0;
tooltip.style.display = 'none';
}
window.removeEventListener('mousemove', this.repositionTooltip)
}
/**
* The current ticks' limits are set to treat iPad (regular/Mini/Pro) as a regular screen.
* @param {*} chart - The chart instance.
* @param {*} dimensions - An object containing the new dimensions *of the chart.*
*/
updateWindowDimensions(chart, dimensions) {
chart.options.scales.x.ticks.maxTicksLimit = dimensions.width < 720 ? 5 : 8
}
maybeHopToHoveredPeriod(e) {
const element = this.chart.getElementsAtEventForMode(e, 'index', { intersect: false })[0]
const date = this.props.graphData.labels[element.index] || this.props.graphData.comparison_labels[element.index]
if (this.props.graphData.interval === 'month') {
navigateToQuery(this.props.history, this.props.query, { period: 'month', date })
} else if (this.props.graphData.interval === 'date') {
navigateToQuery(this.props.history, this.props.query, { period: 'day', date })
}
}
render() {
const { graphData } = this.props
const canvasClass = classNames('mt-4 select-none', { 'cursor-pointer': !['minute', 'hour'].includes(graphData?.interval) })
return (
<FadeIn show={graphData}>
<div className="relative h-96 print:h-auto print:pb-8 w-full z-0">
<canvas id="main-graph-canvas" className={canvasClass}></canvas>
</div>
</FadeIn>
)
}
}
export default withRouter(LineGraph)

@ -0,0 +1,17 @@
import React from "react"
export default function SamplingNotice({topStatData}) {
const samplePercent = topStatData?.samplePercent
if (samplePercent && samplePercent < 100) {
return (
<div tooltip={`Stats based on a ${samplePercent}% sample of all visitors`} className="cursor-pointer w-4 h-4 mx-2">
<svg className="absolute w-4 h-4 dark:text-gray-300 text-gray-700" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth="2" d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
</div>
)
} else {
return null
}
}

@ -0,0 +1,53 @@
import React, { useState } from "react"
import * as api from '../../api'
import { getCurrentInterval } from "./interval-picker"
export default function StatsExport({site, query}) {
const [exporting, setExporting] = useState(false)
function startExport() {
setExporting(true)
document.cookie = "exporting="
pollExportReady()
}
function pollExportReady() {
if (document.cookie.includes('exporting')) {
setTimeout(pollExportReady, 1000)
} else {
setExporting(false)
}
}
function renderLoading() {
return (
<svg className="animate-spin h-4 w-4 text-indigo-500" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24">
<circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4"></circle>
<path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"></path>
</svg>
)
}
function renderExportLink() {
const interval = getCurrentInterval(site, query)
const queryParams = api.serializeQuery(query, [{ interval }])
const endpoint = `/${encodeURIComponent(site.domain)}/export${queryParams}`
return (
<a href={endpoint} download onClick={startExport}>
<svg className="absolute text-gray-700 feather dark:text-gray-300 hover:text-indigo-600 dark:hover:text-indigo-600" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"></path>
<polyline points="7 10 12 15 17 10"></polyline>
<line x1="12" y1="15" x2="12" y2="3"></line>
</svg>
</a>
)
}
return (
<div className="w-4 h-4 mx-2">
{exporting && renderLoading()}
{!exporting && renderExportLink()}
</div>
)
}

@ -3,8 +3,9 @@ import { Tooltip } from '../../util/tooltip'
import { SecondsSinceLastLoad } from '../../util/seconds-since-last-load'
import classNames from "classnames";
import numberFormatter, { durationFormatter } from '../../util/number-formatter'
import { METRIC_MAPPING } from './graph-util'
import * as storage from '../../util/storage'
import { formatDateRange } from '../../util/date.js'
import { getGraphableMetrics } from "./graph-util.js";
function Maybe({condition, children}) {
if (condition) {
@ -14,103 +15,103 @@ function Maybe({condition, children}) {
}
}
export default class TopStats extends React.Component {
renderPercentageComparison(name, comparison, forceDarkBg = false) {
const formattedComparison = numberFormatter(Math.abs(comparison))
const defaultClassName = classNames({
"pl-2 text-xs dark:text-gray-100": !forceDarkBg,
"pl-2 text-xs text-gray-100": forceDarkBg
})
const noChangeClassName = classNames({
"pl-2 text-xs text-gray-700 dark:text-gray-300": !forceDarkBg,
"pl-2 text-xs text-gray-300": forceDarkBg
})
if (comparison > 0) {
const color = name === 'Bounce rate' ? 'text-red-400' : 'text-green-500'
return <span className={defaultClassName}><span className={color + ' font-bold'}>&uarr;</span> {formattedComparison}%</span>
} else if (comparison < 0) {
const color = name === 'Bounce rate' ? 'text-green-500' : 'text-red-400'
return <span className={defaultClassName}><span className={color + ' font-bold'}>&darr;</span> {formattedComparison}%</span>
} else if (comparison === 0) {
return <span className={noChangeClassName}>&#12336; 0%</span>
} else {
return null
}
function renderPercentageComparison(name, comparison, forceDarkBg = false) {
const formattedComparison = numberFormatter(Math.abs(comparison))
const defaultClassName = classNames({
"pl-2 text-xs dark:text-gray-100": !forceDarkBg,
"pl-2 text-xs text-gray-100": forceDarkBg
})
const noChangeClassName = classNames({
"pl-2 text-xs text-gray-700 dark:text-gray-300": !forceDarkBg,
"pl-2 text-xs text-gray-300": forceDarkBg
})
if (comparison > 0) {
const color = name === 'Bounce rate' ? 'text-red-400' : 'text-green-500'
return <span className={defaultClassName}><span className={color + ' font-bold'}>&uarr;</span> {formattedComparison}%</span>
} else if (comparison < 0) {
const color = name === 'Bounce rate' ? 'text-green-500' : 'text-red-400'
return <span className={defaultClassName}><span className={color + ' font-bold'}>&darr;</span> {formattedComparison}%</span>
} else if (comparison === 0) {
return <span className={noChangeClassName}>&#12336; 0%</span>
} else {
return null
}
}
topStatNumberShort(name, value) {
if (['visit duration', 'time on page'].includes(name.toLowerCase())) {
return durationFormatter(value)
} else if (['bounce rate', 'conversion rate'].includes(name.toLowerCase())) {
return value + '%'
} else if (['average revenue', 'total revenue'].includes(name.toLowerCase())) {
return value?.short
} else {
return numberFormatter(value)
}
function topStatNumberShort(name, value) {
if (['visit duration', 'time on page'].includes(name.toLowerCase())) {
return durationFormatter(value)
} else if (['bounce rate', 'conversion rate'].includes(name.toLowerCase())) {
return value + '%'
} else if (['average revenue', 'total revenue'].includes(name.toLowerCase())) {
return value?.short
} else {
return numberFormatter(value)
}
}
topStatNumberLong(name, value) {
if (['visit duration', 'time on page'].includes(name.toLowerCase())) {
return durationFormatter(value)
} else if (['bounce rate', 'conversion rate'].includes(name.toLowerCase())) {
return value + '%'
} else if (['average revenue', 'total revenue'].includes(name.toLowerCase())) {
return value?.long
} else {
return (value || 0).toLocaleString()
}
function topStatNumberLong(name, value) {
if (['visit duration', 'time on page'].includes(name.toLowerCase())) {
return durationFormatter(value)
} else if (['bounce rate', 'conversion rate'].includes(name.toLowerCase())) {
return value + '%'
} else if (['average revenue', 'total revenue'].includes(name.toLowerCase())) {
return value?.long
} else {
return (value || 0).toLocaleString()
}
}
export default function TopStats(props) {
const {site, query, data, onMetricUpdate, tooltipBoundary, lastLoadTimestamp} = props
topStatTooltip(stat, query) {
function tooltip(stat) {
let statName = stat.name.toLowerCase()
statName = stat.value === 1 ? statName.slice(0, -1) : statName
const { topStatData, lastLoadTimestamp } = this.props
const showingImported = topStatData?.imported_source && topStatData?.with_imported
return (
<div>
{query.comparison && <div className="whitespace-nowrap">
{this.topStatNumberLong(stat.name, stat.value)} vs. {this.topStatNumberLong(stat.name, stat.comparison_value)} {statName}
<span className="ml-2">{this.renderPercentageComparison(stat.name, stat.change, true)}</span>
{topStatNumberLong(stat.name, stat.value)} vs. {topStatNumberLong(stat.name, stat.comparison_value)} {statName}
<span className="ml-2">{renderPercentageComparison(stat.name, stat.change, true)}</span>
</div>}
{!query.comparison && <div className="whitespace-nowrap">
{this.topStatNumberLong(stat.name, stat.value)} {statName}
{topStatNumberLong(stat.name, stat.value)} {statName}
</div>}
{stat.name === 'Current visitors' && <p className="font-normal text-xs">Last updated <SecondsSinceLastLoad lastLoadTimestamp={lastLoadTimestamp}/>s ago</p>}
{stat.name === 'Views per visit' && showingImported && <p className="font-normal text-xs whitespace-nowrap">Based only on native data</p>}
</div>
)
}
canMetricBeGraphed(stat) {
const isTotalUniqueVisitors = this.props.query.filters.goal && stat.name === 'Unique visitors'
const isKnownMetric = Object.keys(METRIC_MAPPING).includes(stat.name)
return isKnownMetric && !isTotalUniqueVisitors
function canMetricBeGraphed(stat) {
const graphableMetrics = getGraphableMetrics(query, site)
return stat.graph_metric && graphableMetrics.includes(stat.graph_metric)
}
maybeUpdateMetric(stat) {
if (this.canMetricBeGraphed(stat)) {
this.props.updateMetric(METRIC_MAPPING[stat.name])
function maybeUpdateMetric(stat) {
if (canMetricBeGraphed(stat)) {
storage.setItem(`metric__${site.domain}`, stat.graph_metric)
onMetricUpdate(stat.graph_metric)
}
}
blinkingDot() {
function blinkingDot() {
return (
<div key="dot" className="block pulsating-circle" style={{ left: '125px', top: '52px' }}></div>
)
}
renderStatName(stat) {
const { metric } = this.props
const isSelected = metric === METRIC_MAPPING[stat.name]
function getStoredMetric() {
return storage.getItem(`metric__${site.domain}`)
}
function renderStatName(stat) {
const isSelected = stat.graph_metric === getStoredMetric()
const [statDisplayName, statExtraName] = stat.name.split(/(\(.+\))/g)
@ -127,48 +128,45 @@ export default class TopStats extends React.Component {
)
}
render() {
const { topStatData, query, site } = this.props
const stats = topStatData && topStatData.top_stats.map((stat, index) => {
const className = classNames('px-4 md:px-6 w-1/2 my-4 lg:w-auto group select-none', {
'cursor-pointer': this.canMetricBeGraphed(stat),
'lg:border-l border-gray-300': index > 0,
'border-r lg:border-r-0': index % 2 === 0
})
return (
<Tooltip key={stat.name} info={this.topStatTooltip(stat, query)} className={className} onClick={() => { this.maybeUpdateMetric(stat) }} boundary={this.props.tooltipBoundary}>
{this.renderStatName(stat)}
<div className="my-1 space-y-2">
<div>
<span className="flex items-center justify-between whitespace-nowrap">
<p className="font-bold text-xl dark:text-gray-100" id={METRIC_MAPPING[stat.name]}>{this.topStatNumberShort(stat.name, stat.value)}</p>
<Maybe condition={!query.comparison}>
{ this.renderPercentageComparison(stat.name, stat.change) }
</Maybe>
</span>
<Maybe condition={query.comparison}>
<p className="text-xs dark:text-gray-100">{ formatDateRange(site, topStatData.from, topStatData.to) }</p>
</Maybe>
</div>
function renderStat(stat, index) {
const className = classNames('px-4 md:px-6 w-1/2 my-4 lg:w-auto group select-none', {
'cursor-pointer': canMetricBeGraphed(stat),
'lg:border-l border-gray-300': index > 0,
'border-r lg:border-r-0': index % 2 === 0
})
return (
<Tooltip key={stat.name} info={tooltip(stat, query)} className={className} onClick={() => { maybeUpdateMetric(stat) }} boundary={tooltipBoundary}>
{renderStatName(stat)}
<div className="my-1 space-y-2">
<div>
<span className="flex items-center justify-between whitespace-nowrap">
<p className="font-bold text-xl dark:text-gray-100" id={stat.graph_metric}>{topStatNumberShort(stat.name, stat.value)}</p>
<Maybe condition={!query.comparison}>
{ renderPercentageComparison(stat.name, stat.change) }
</Maybe>
</span>
<Maybe condition={query.comparison}>
<div>
<p className="font-bold text-xl text-gray-500 dark:text-gray-400">{ this.topStatNumberShort(stat.name, stat.comparison_value) }</p>
<p className="text-xs text-gray-500 dark:text-gray-400">{ formatDateRange(site, topStatData.comparing_from, topStatData.comparing_to) }</p>
</div>
<p className="text-xs dark:text-gray-100">{ formatDateRange(site, data.from, data.to) }</p>
</Maybe>
</div>
<Maybe condition={query.comparison}>
<div>
<p className="font-bold text-xl text-gray-500 dark:text-gray-400">{ topStatNumberShort(stat.name, stat.comparison_value) }</p>
<p className="text-xs text-gray-500 dark:text-gray-400">{ formatDateRange(site, data.comparing_from, data.comparing_to) }</p>
</div>
</Tooltip>
)
})
</Maybe>
</div>
</Tooltip>
)
}
if (stats && query && query.period === 'realtime') {
stats.push(this.blinkingDot())
}
const stats = data && data.top_stats.map(renderStat)
return stats || null;
if (stats && query.period === 'realtime') {
stats.push(blinkingDot())
}
return stats || null;
}

@ -1,534 +1,153 @@
import React from 'react';
import { withRouter, Link } from 'react-router-dom'
import Chart from 'chart.js/auto';
import { navigateToQuery } from '../../query'
import React, { useState, useEffect, useRef, useCallback } from 'react';
import * as api from '../../api'
import * as storage from '../../util/storage'
import LazyLoader from '../../components/lazy-loader'
import GraphTooltip from './graph-tooltip'
import { buildDataSet, METRIC_MAPPING, METRIC_LABELS, METRIC_FORMATTER, LoadingState } from './graph-util'
import dateFormatter from './date-formatter';
import { getGraphableMetrics } from './graph-util'
import TopStats from './top-stats';
import { IntervalPicker, getStoredInterval, storeInterval } from './interval-picker';
import { IntervalPicker, getCurrentInterval } from './interval-picker'
import StatsExport from './stats-export'
import WithImportedSwitch from './with-imported-switch';
import SamplingNotice from './sampling-notice';
import FadeIn from '../../fade-in';
import * as url from '../../util/url'
import classNames from 'classnames';
import { monthsBetweenDates, parseNaiveDate, isBefore } from '../../util/date'
import { isComparisonEnabled } from '../../comparison-input'
import LineGraphWithRouter from './line-graph'
const calculateMaximumY = function(dataset) {
const yAxisValues = dataset
.flatMap((item) => item.data)
.map((item) => item || 0)
if (yAxisValues) {
return Math.max(...yAxisValues)
} else {
return 1
}
}
class LineGraph extends React.Component {
constructor(props) {
super(props);
this.boundary = React.createRef()
this.regenerateChart = this.regenerateChart.bind(this);
this.updateWindowDimensions = this.updateWindowDimensions.bind(this);
this.state = {
exported: false
};
function fetchTopStats(site, query) {
const q = { ...query }
if (!isComparisonEnabled(q.comparison)) {
q.comparison = 'previous_period'
}
regenerateChart() {
const { graphData, metric, query } = this.props
const graphEl = document.getElementById("main-graph-canvas")
this.ctx = graphEl.getContext('2d');
const dataSet = buildDataSet(graphData.plot, graphData.comparison_plot, graphData.present_index, this.ctx, METRIC_LABELS[metric])
return new Chart(this.ctx, {
type: 'line',
data: {
labels: graphData.labels,
datasets: dataSet
},
options: {
animation: false,
plugins: {
legend: { display: false },
tooltip: {
enabled: false,
mode: 'index',
intersect: false,
position: 'average',
external: GraphTooltip(graphData, metric, query)
},
},
responsive: true,
maintainAspectRatio: false,
onResize: this.updateWindowDimensions,
elements: { line: { tension: 0 }, point: { radius: 0 } },
onClick: this.onClick.bind(this),
scale: {
ticks: { precision: 0, maxTicksLimit: 8 }
},
scales: {
y: {
min: 0,
suggestedMax: calculateMaximumY(dataSet),
ticks: {
callback: METRIC_FORMATTER[metric],
color: this.props.darkTheme ? 'rgb(243, 244, 246)' : undefined
},
grid: {
zeroLineColor: 'transparent',
drawBorder: false,
}
},
yComparison: {
min: 0,
suggestedMax: calculateMaximumY(dataSet),
display: false,
grid: { display: false },
},
x: {
grid: { display: false },
ticks: {
callback: function(val, _index, _ticks) {
if (this.getLabelForValue(val) == "__blank__") return ""
const hasMultipleYears =
graphData.labels
.filter((date) => typeof date === 'string')
.map(date => date.split('-')[0])
.filter((value, index, list) => list.indexOf(value) === index)
.length > 1
if (graphData.interval === 'hour' && query.period !== 'day') {
const date = dateFormatter({
interval: "date",
longForm: false,
period: query.period,
shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
const hour = dateFormatter({
interval: graphData.interval,
longForm: false,
period: query.period,
shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
return api.get(url.apiPath(site, '/top-stats'), q)
}
// Returns a combination of date and hour. This is because
// small intervals like hour may return multiple days
// depending on the query period.
return `${date}, ${hour}`
}
function fetchMainGraph(site, query, metric, interval) {
const params = {metric, interval}
return api.get(url.apiPath(site, '/main-graph'), query, params)
}
if (graphData.interval === 'minute' && query.period !== 'realtime') {
return dateFormatter({
interval: "hour", longForm: false, period: query.period,
})(this.getLabelForValue(val))
}
export default function VisitorGraph(props) {
const {site, query, lastLoadTimestamp} = props
const isRealtime = query.period === 'realtime'
const isDarkTheme = document.querySelector('html').classList.contains('dark') || false
return dateFormatter({
interval: graphData.interval, longForm: false, period: query.period, shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
},
color: this.props.darkTheme ? 'rgb(243, 244, 246)' : undefined
}
}
},
interaction: {
mode: 'index',
intersect: false,
}
}
});
}
repositionTooltip(e) {
const tooltipEl = document.getElementById('chartjs-tooltip');
if (tooltipEl && window.innerWidth >= 768) {
if (e.clientX > 0.66 * window.innerWidth) {
tooltipEl.style.right = (window.innerWidth - e.clientX) + window.pageXOffset + 'px'
tooltipEl.style.left = null;
} else {
tooltipEl.style.right = null;
tooltipEl.style.left = e.clientX + window.pageXOffset + 'px'
}
tooltipEl.style.top = e.clientY + window.pageYOffset + 'px'
tooltipEl.style.opacity = 1;
}
}
const topStatsBoundary = useRef(null)
componentDidMount() {
if (this.props.graphData) {
this.chart = this.regenerateChart();
}
window.addEventListener('mousemove', this.repositionTooltip);
}
const [topStatData, setTopStatData] = useState(null)
const [topStatsLoading, setTopStatsLoading] = useState(true)
const [graphData, setGraphData] = useState(null)
const [graphLoading, setGraphLoading] = useState(true)
componentDidUpdate(prevProps) {
const { graphData, darkTheme } = this.props;
const tooltip = document.getElementById('chartjs-tooltip');
// This state is explicitly meant for the situation where either graph interval
// or graph metric is changed. That results in behaviour where Top Stats stay
// intact, but the graph container alone will display a loading spinner for as
// long as new graph data is fetched.
const [graphRefreshing, setGraphRefreshing] = useState(false)
if (
graphData !== prevProps.graphData ||
darkTheme !== prevProps.darkTheme
) {
if (graphData) {
if (this.chart) {
this.chart.destroy();
}
this.chart = this.regenerateChart();
this.chart.update();
}
const onIntervalUpdate = useCallback((newInterval) => {
setGraphData(null)
setGraphRefreshing(true)
fetchGraphData(getStoredMetric(), newInterval)
}, [query])
if (tooltip) {
tooltip.style.display = 'none';
}
}
const onMetricUpdate = useCallback((newMetric) => {
setGraphData(null)
setGraphRefreshing(true)
fetchGraphData(newMetric, getCurrentInterval(site, query))
}, [query])
if (!graphData) {
if (this.chart) {
this.chart.destroy();
}
useEffect(() => {
setTopStatData(null)
setTopStatsLoading(true)
setGraphData(null)
setGraphLoading(true)
fetchTopStatsAndGraphData()
if (tooltip) {
tooltip.style.display = 'none';
}
if (isRealtime) {
document.addEventListener('tick', fetchTopStatsAndGraphData)
}
}
componentWillUnmount() {
// Ensure that the tooltip doesn't hang around when we are loading more data
const tooltip = document.getElementById('chartjs-tooltip');
if (tooltip) {
tooltip.style.opacity = 0;
tooltip.style.display = 'none';
return () => {
document.removeEventListener('tick', fetchTopStatsAndGraphData)
}
window.removeEventListener('mousemove', this.repositionTooltip)
}
/**
* The current ticks' limits are set to treat iPad (regular/Mini/Pro) as a regular screen.
* @param {*} chart - The chart instance.
* @param {*} dimensions - An object containing the new dimensions *of the chart.*
*/
updateWindowDimensions(chart, dimensions) {
chart.options.scales.x.ticks.maxTicksLimit = dimensions.width < 720 ? 5 : 8
}
}, [query])
onClick(e) {
const element = this.chart.getElementsAtEventForMode(e, 'index', { intersect: false })[0]
const date = this.props.graphData.labels[element.index] || this.props.graphData.comparison_labels[element.index]
useEffect(() => {
if (topStatData) { storeTopStatsContainerHeight() }
}, [topStatData])
if (this.props.graphData.interval === 'month') {
navigateToQuery(this.props.history, this.props.query, { period: 'month', date })
} else if (this.props.graphData.interval === 'date') {
navigateToQuery(this.props.history, this.props.query, { period: 'day', date })
function fetchTopStatsAndGraphData() {
fetchTopStats(site, query)
.then((res) => {
setTopStatData(res)
setTopStatsLoading(false)
})
let metric = getStoredMetric()
const availableMetrics = getGraphableMetrics(query, site)
if (!availableMetrics.includes(metric)) {
metric = availableMetrics[0]
storage.setItem(`metric__${site.domain}`, metric)
}
}
pollExportReady() {
if (document.cookie.includes('exporting')) {
setTimeout(this.pollExportReady.bind(this), 1000);
} else {
this.setState({ exported: false })
}
}
const interval = getCurrentInterval(site, query)
downloadSpinner() {
this.setState({ exported: true });
document.cookie = "exporting=";
setTimeout(this.pollExportReady.bind(this), 1000);
fetchGraphData(metric, interval)
}
downloadLink() {
if (this.props.query.period !== 'realtime') {
if (this.state.exported) {
return (
<div className="w-4 h-4 mx-2">
<svg className="animate-spin h-4 w-4 text-indigo-500" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24">
<circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4"></circle>
<path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"></path>
</svg>
</div>
)
} else {
const interval = this.props.graphData?.interval
const queryParams = api.serializeQuery(this.props.query, [{ interval }])
const endpoint = `/${encodeURIComponent(this.props.site.domain)}/export${queryParams}`
return (
<a className="w-4 h-4 mx-2" href={endpoint} download onClick={this.downloadSpinner.bind(this)}>
<svg className="absolute text-gray-700 feather dark:text-gray-300 hover:text-indigo-600 dark:hover:text-indigo-600" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"></path><polyline points="7 10 12 15 17 10"></polyline><line x1="12" y1="15" x2="12" y2="3"></line></svg>
</a>
)
}
}
function fetchGraphData(metric, interval) {
fetchMainGraph(site, query, metric, interval)
.then((res) => {
setGraphData(res)
setGraphLoading(false)
setGraphRefreshing(false)
})
}
samplingNotice() {
const samplePercent = this.props.topStatData && this.props.topStatData.sample_percent
if (samplePercent < 100) {
return (
<div tooltip={`Stats based on a ${samplePercent}% sample of all visitors`} className="cursor-pointer w-4 h-4 mx-2">
<svg className="absolute w-4 h-4 dark:text-gray-300 text-gray-700" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth="2" d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
</div>
)
}
function getStoredMetric() {
return storage.getItem(`metric__${site.domain}`)
}
importedNotice() {
if (!this.props.topStatData?.imported_source) return
const isBeforeNativeStats = (date) => {
if (!date) return false
const nativeStatsBegin = parseNaiveDate(this.props.site.nativeStatsBegin)
const parsedDate = parseNaiveDate(date)
return isBefore(parsedDate, nativeStatsBegin, "day")
}
const isQueryingImportedPeriod = isBeforeNativeStats(this.props.topStatData.from)
const isComparingImportedPeriod = isBeforeNativeStats(this.props.topStatData.comparing_from)
if (isQueryingImportedPeriod || isComparingImportedPeriod) {
const source = this.props.topStatData.imported_source
const withImported = this.props.topStatData.with_imported;
const strike = withImported ? "" : " line-through"
const target = url.setQuery('with_imported', !withImported)
const tip = withImported ? "" : "do not ";
return (
<Link to={target} className="w-4 h-4 mx-2">
<div tooltip={`Stats ${tip}include data imported from ${source}.`} className="cursor-pointer w-4 h-4">
<svg className="absolute dark:text-gray-300 text-gray-700" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<text x="4" y="18" fontSize="24" fill="currentColor" className={"text-gray-700 dark:text-gray-300" + strike}>{source[0].toUpperCase()}</text>
</svg>
</div>
</Link>
)
}
function storeTopStatsContainerHeight() {
storage.setItem(`topStatsHeight__${site.domain}`, document.getElementById('top-stats-container').clientHeight)
}
// This function is used for maintaining the main-graph/top-stats container height in the
// loading process. The container height depends on how many top stat metrics are returned
// from the API, but in the loading state, we don't know that yet. We can use localStorage
// to keep track of the Top Stats container height.
getTopStatsHeight() {
if (this.props.topStatData) {
function getTopStatsHeight() {
if (topStatData) {
return 'auto'
} else {
return `${storage.getItem(`topStatsHeight__${this.props.site.domain}`) || 89}px`
return `${storage.getItem(`topStatsHeight__${site.domain}`) || 89}px`
}
}
render() {
const { mainGraphRefreshing, updateMetric, updateInterval, metric, topStatData, query, site, graphData, lastLoadTimestamp } = this.props
const canvasClass = classNames('mt-4 select-none', { 'cursor-pointer': !['minute', 'hour'].includes(graphData?.interval) })
return (
<div>
<div id="top-stats-container" className="flex flex-wrap" ref={this.boundary} style={{ height: this.getTopStatsHeight() }}>
<TopStats site={site} query={query} metric={metric} updateMetric={updateMetric} topStatData={topStatData} tooltipBoundary={this.boundary.current} lastLoadTimestamp={lastLoadTimestamp} />
return (
<div className={"relative w-full mt-2 bg-white rounded shadow-xl dark:bg-gray-825"}>
{(topStatsLoading || graphLoading) && renderLoader()}
<FadeIn show={!(topStatsLoading || graphLoading)}>
<div id="top-stats-container" className="flex flex-wrap" ref={topStatsBoundary} style={{ height: getTopStatsHeight() }}>
<TopStats site={site} query={query} data={topStatData} onMetricUpdate={onMetricUpdate} tooltipBoundary={topStatsBoundary.current} lastLoadTimestamp={lastLoadTimestamp} />
</div>
<div className="relative px-2">
{mainGraphRefreshing && renderLoader()}
{graphRefreshing && renderLoader()}
<div className="absolute right-4 -top-8 py-1 flex items-center">
{this.downloadLink()}
{this.samplingNotice()}
{this.importedNotice()}
<IntervalPicker site={site} query={query} graphData={graphData} metric={metric} updateInterval={updateInterval} />
{!isRealtime && <StatsExport site={site} query={query} />}
<SamplingNotice samplePercent={topStatData}/>
<WithImportedSwitch site={site} topStatData={topStatData} />
<IntervalPicker site={site} query={query} onIntervalUpdate={onIntervalUpdate} />
</div>
<FadeIn show={graphData}>
<div className="relative h-96 w-full z-0">
<canvas id="main-graph-canvas" className={canvasClass}></canvas>
</div>
</FadeIn>
<LineGraphWithRouter graphData={graphData} darkTheme={isDarkTheme} query={query} />
</div>
</div>
)
}
}
const LineGraphWithRouter = withRouter(LineGraph)
export default class VisitorGraph extends React.Component {
constructor(props) {
super(props)
this.state = {
topStatsLoadingState: LoadingState.loading,
mainGraphLoadingState: LoadingState.loading,
metric: storage.getItem(`metric__${this.props.site.domain}`) || 'visitors'
}
this.onVisible = this.onVisible.bind(this)
this.updateMetric = this.updateMetric.bind(this)
this.fetchTopStatData = this.fetchTopStatData.bind(this)
this.fetchGraphData = this.fetchGraphData.bind(this)
this.updateInterval = this.updateInterval.bind(this)
}
isIntervalValid(interval) {
const { query, site } = this.props
const validIntervals = site.validIntervalsByPeriod[query.period] || []
return validIntervals.includes(interval)
}
getIntervalFromStorage() {
const { query, site } = this.props
let interval = getStoredInterval(query.period, site.domain)
if (interval !== "week" && interval !== "month" && query.period === "custom" && monthsBetweenDates(query.from, query.to) > 12) {
interval = "month"
}
if (this.isIntervalValid(interval)) {
return interval
} else {
return null
}
}
updateInterval(interval) {
if (this.isIntervalValid(interval)) {
storeInterval(this.props.query.period, this.props.site.domain, interval)
this.setState({ mainGraphLoadingState: LoadingState.refreshing, graphData: null }, this.fetchGraphData)
}
}
onVisible() {
this.setState({ mainGraphLoadingState: LoadingState.loading }, this.fetchGraphData)
this.fetchTopStatData()
if (this.props.query.period === 'realtime') {
document.addEventListener('tick', this.fetchGraphData)
document.addEventListener('tick', this.fetchTopStatData)
}
}
componentDidUpdate(prevProps, prevState) {
const { metric } = this.state;
const { query } = this.props
if (query !== prevProps.query) {
this.setState({ mainGraphLoadingState: LoadingState.loading, topStatsLoadingState: LoadingState.loading, graphData: null, topStatData: null }, this.fetchGraphData)
this.fetchTopStatData()
}
if (metric !== prevState.metric) {
this.setState({ mainGraphLoadingState: LoadingState.refreshing }, this.fetchGraphData)
}
}
resetMetric() {
const { topStatData } = this.state
const { query, site } = this.props
const savedMetric = storage.getItem(`metric__${site.domain}`)
const selectableMetrics = topStatData && topStatData.top_stats.map(({ name }) => METRIC_MAPPING[name]).filter(name => name)
const canSelectSavedMetric = selectableMetrics && selectableMetrics.includes(savedMetric)
if (query.filters.goal) {
this.setState({ metric: 'conversions' })
} else if (canSelectSavedMetric) {
this.setState({ metric: savedMetric })
} else {
this.setState({ metric: 'visitors' })
}
}
componentWillUnmount() {
document.removeEventListener('tick', this.fetchGraphData)
document.removeEventListener('tick', this.fetchTopStatData)
}
storeTopStatsContainerHeight() {
storage.setItem(`topStatsHeight__${this.props.site.domain}`, document.getElementById('top-stats-container').clientHeight)
}
updateMetric(clickedMetric) {
if (this.state.metric == clickedMetric) return
storage.setItem(`metric__${this.props.site.domain}`, clickedMetric)
this.setState({ metric: clickedMetric, graphData: null })
}
fetchGraphData() {
const url = `/api/stats/${encodeURIComponent(this.props.site.domain)}/main-graph`
let params = { metric: this.state.metric }
const interval = this.getIntervalFromStorage()
if (interval) { params.interval = interval }
api.get(url, this.props.query, params)
.then((res) => {
this.setState({ mainGraphLoadingState: LoadingState.loaded, graphData: res })
return res
})
.catch((err) => {
console.log(err)
this.setState({ mainGraphLoadingState: LoadingState.loaded, graphData: false })
})
}
fetchTopStatData() {
const query = { ...this.props.query }
if (!isComparisonEnabled(query.comparison)) query.comparison = 'previous_period'
api.get(`/api/stats/${encodeURIComponent(this.props.site.domain)}/top-stats`, query)
.then((res) => {
this.setState({ topStatsLoadingState: LoadingState.loaded, topStatData: res }, () => {
this.storeTopStatsContainerHeight()
this.resetMetric()
})
return res
})
}
renderInner() {
const { query, site } = this.props;
const { graphData, metric, topStatData, topStatsLoadingState, mainGraphLoadingState } = this.state;
const theme = document.querySelector('html').classList.contains('dark') || false
const mainGraphRefreshing = (mainGraphLoadingState === LoadingState.refreshing)
const topStatAndGraphLoaded = !!(topStatData && graphData)
const shouldShow =
topStatsLoadingState === LoadingState.loaded &&
LoadingState.isLoadedOrRefreshing(mainGraphLoadingState) &&
(topStatData && mainGraphRefreshing || topStatAndGraphLoaded)
return (
<FadeIn show={shouldShow}>
<LineGraphWithRouter mainGraphRefreshing={mainGraphRefreshing} graphData={graphData} topStatData={topStatData} site={site} query={query} darkTheme={theme} metric={metric} updateMetric={this.updateMetric} updateInterval={this.updateInterval} lastLoadTimestamp={this.props.lastLoadTimestamp} />
</FadeIn>
)
}
render() {
const { mainGraphLoadingState, topStatsLoadingState } = this.state
const showLoader =
[mainGraphLoadingState, topStatsLoadingState].includes(LoadingState.loading) &&
mainGraphLoadingState !== LoadingState.refreshing
return (
<LazyLoader onVisible={this.onVisible}>
<div className={"relative w-full mt-2 bg-white rounded shadow-xl dark:bg-gray-825"}>
{showLoader && renderLoader()}
{this.renderInner()}
</div>
</LazyLoader>
)
}
</div>
)
}
function renderLoader() {

@ -0,0 +1,40 @@
import React from "react"
import { parseNaiveDate, isBefore } from '../../util/date'
import { Link } from 'react-router-dom'
import * as url from '../../util/url'
import { BarsArrowUpIcon } from '@heroicons/react/20/solid'
export default function WithImportedSwitch({site, topStatData}) {
if (!topStatData?.imports_exist) {
return null
}
function isBeforeNativeStats(date) {
if (!date) return false
const nativeStatsBegin = parseNaiveDate(site.nativeStatsBegin)
const parsedDate = parseNaiveDate(date)
return isBefore(parsedDate, nativeStatsBegin, "day")
}
const isQueryingImportedPeriod = isBeforeNativeStats(topStatData.from)
const isComparingImportedPeriod = isBeforeNativeStats(topStatData.comparing_from)
if (isQueryingImportedPeriod || isComparingImportedPeriod) {
const withImported = topStatData.with_imported;
const toggleColor = withImported ? " dark:text-gray-300 text-gray-700" : " dark:text-gray-500 text-gray-400"
const target = url.setQuery('with_imported', !withImported)
const tip = withImported ? "" : "do not ";
return (
<Link to={target} className="w-4 h-4 mx-2">
<div tooltip={`Stats ${tip}include imported data.`} className="cursor-pointer w-4 h-4">
<BarsArrowUpIcon className={"absolute " + toggleColor} />
</div>
</Link>
)
} else {
return null
}
}

@ -57,7 +57,7 @@ function Regions({query, site, onClick}) {
getFilterFor={getFilterFor}
onClick={onClick}
keyLabel="Region"
metrics={[VISITORS_METRIC]}
metrics={maybeWithCR([VISITORS_METRIC], query)}
detailsLink={sitePath(site, '/regions')}
query={query}
renderIcon={renderIcon}
@ -84,7 +84,7 @@ function Cities({query, site}) {
fetchData={fetchData}
getFilterFor={getFilterFor}
keyLabel="City"
metrics={[VISITORS_METRIC]}
metrics={maybeWithCR([VISITORS_METRIC], query)}
detailsLink={sitePath(site, '/cities')}
query={query}
renderIcon={renderIcon}

@ -1,15 +1,26 @@
import React, { useEffect, useState } from "react";
import { Link } from 'react-router-dom'
import { withRouter } from 'react-router-dom'
import Money from "../behaviours/money";
import Modal from './modal'
import * as api from '../../api'
import * as url from "../../util/url";
import numberFormatter from '../../util/number-formatter'
import {parseQuery} from '../../query'
import { parseQuery } from '../../query'
import { escapeFilterValue } from '../../util/filters'
/*global BUILD_EXTRA*/
/*global require*/
function maybeRequire() {
if (BUILD_EXTRA) {
return require('../../extra/money')
} else {
return { default: null }
}
}
const Money = maybeRequire().default
function ConversionsModal(props) {
const site = props.site
const query = parseQuery(props.location.search, site)
@ -24,7 +35,7 @@ function ConversionsModal(props) {
}, [])
function fetchData() {
api.get(url.apiPath(site, `/conversions`), query, {limit: 100, page})
api.get(url.apiPath(site, `/conversions`), query, { limit: 100, page })
.then((res) => {
setLoading(false)
setList(list.concat(res))
@ -59,16 +70,16 @@ function ConversionsModal(props) {
<tr className="text-sm dark:text-gray-200" key={listItem.name}>
<td className="p-2">
<Link
to={{pathname: url.siteBasePath(site), search: filterSearchLink(listItem)}}
to={{ pathname: url.siteBasePath(site), search: filterSearchLink(listItem) }}
className="hover:underline block truncate">
{listItem.name}
{listItem.name}
</Link>
</td>
<td className="p-2 w-24 font-medium" align="right">{numberFormatter(listItem.visitors)}</td>
<td className="p-2 w-24 font-medium" align="right">{numberFormatter(listItem.events)}</td>
<td className="p-2 w-24 font-medium" align="right">{listItem.conversion_rate}%</td>
{ hasRevenue && <td className="p-2 w-24 font-medium" align="right"><Money formatted={listItem.total_revenue}/></td> }
{ hasRevenue && <td className="p-2 w-24 font-medium" align="right"><Money formatted={listItem.average_revenue}/></td> }
{hasRevenue && <td className="p-2 w-24 font-medium" align="right"><Money formatted={listItem.total_revenue} /></td>}
{hasRevenue && <td className="p-2 w-24 font-medium" align="right"><Money formatted={listItem.average_revenue} /></td>}
</tr>
)
}
@ -78,7 +89,7 @@ function ConversionsModal(props) {
}
function renderBody() {
const hasRevenue = list.some((goal) => goal.total_revenue)
const hasRevenue = BUILD_EXTRA && list.some((goal) => goal.total_revenue)
return (
<>
@ -98,7 +109,7 @@ function ConversionsModal(props) {
</tr>
</thead>
<tbody>
{ list.map((item) => renderListItem(item, hasRevenue)) }
{list.map((item) => renderListItem(item, hasRevenue))}
</tbody>
</table>
</main>
@ -108,9 +119,9 @@ function ConversionsModal(props) {
return (
<Modal site={site}>
{ renderBody() }
{ loading && renderLoading() }
{ !loading && moreResultsAvailable && renderLoadMore() }
{renderBody()}
{loading && renderLoading()}
{!loading && moreResultsAvailable && renderLoadMore()}
</Modal>
)
}

@ -1,30 +1,32 @@
import React, { useCallback, useEffect, useState } from 'react'
import React, { useEffect, useState, useMemo } from 'react'
import { withRouter } from "react-router-dom";
import Combobox from '../../components/combobox'
import FilterTypeSelector from "../../components/filter-type-selector";
import { FILTER_TYPES } from "../../util/filters";
import { FILTER_OPERATIONS } from "../../util/filters";
import { parseQuery } from '../../query'
import * as api from '../../api'
import { apiPath, siteBasePath } from '../../util/url'
import { toFilterQuery, parseQueryFilter } from '../../util/filters';
import { siteBasePath } from '../../util/url'
import { toFilterQuery, parseQueryPropsFilter } from '../../util/filters';
import { shouldIgnoreKeypress } from '../../keybinding';
import PropFilterRow from './prop-filter-row';
function getFormState(query) {
const rawValue = query.filters['props']
if (rawValue) {
const [[propKey, _propValue]] = Object.entries(rawValue)
const {type, clauses} = parseQueryFilter(query, 'props')
if (query.filters['props']) {
const values = Object.fromEntries(parseQueryPropsFilter(query, 'props').map((value, index) => [index, value]))
return {
prop_key: {value: propKey, label: propKey},
prop_value: { type: type, clauses: clauses }
entries: Object.keys(values).sort(),
values
}
} else {
return {
entries: [0],
values: {
0: {
propKey: null,
type: FILTER_OPERATIONS.is,
clauses: []
}
}
}
}
return {
prop_key: null,
prop_value: { type: FILTER_TYPES.is, clauses: [] }
}
}
@ -32,68 +34,78 @@ function PropFilterModal(props) {
const query = parseQuery(props.location.search, props.site)
const [formState, setFormState] = useState(getFormState(query))
function fetchPropKeyOptions() {
return (input) => {
return api.get(apiPath(props.site, "/suggestions/prop_key"), query, { q: input.trim() })
}
const selectedPropKeys = useMemo(() => Object.values(formState.values).map((value) => value.propKey), [formState])
function onPropKeySelect(id, selectedOptions) {
const newPropKey = selectedOptions.length === 0 ? null : selectedOptions[0]
setFormState(prevState => ({
...prevState,
values: {
...prevState.values,
[id]: {
...prevState.values[id],
propKey: newPropKey,
clauses: []
}
}
}))
}
const fetchPropValueOptions = useCallback(() => {
return (input) => {
const propKey = formState.prop_key?.value
const updatedQuery = { ...query, filters: { ...query.filters, props: {[propKey]: '!(none)'} } }
return api.get(apiPath(props.site, "/suggestions/prop_value"), updatedQuery, { q: input.trim() })
}
}, [formState.prop_key])
function onPropKeySelect() {
return (selectedOptions) => {
const newPropKey = selectedOptions.length === 0 ? null : selectedOptions[0]
setFormState(prevState => ({
prop_key: newPropKey,
prop_value: { type: prevState.prop_value.type, clauses: [] }
}))
}
function onPropValueSelect(id, selection) {
setFormState(prevState => ({
...prevState,
values: {
...prevState.values,
[id]: {
...prevState.values[id],
clauses: selection
}
}
}))
}
function onPropValueSelect() {
return (selection) => {
setFormState(prevState => ({
...prevState, prop_value: { ...prevState.prop_value, clauses: selection }
}))
}
function onFilterTypeSelect(id, newType) {
setFormState(prevState => ({
...prevState,
values: {
...prevState.values,
[id]: {
...prevState.values[id],
type: newType
}
}
}))
}
function onFilterTypeSelect() {
return (newType) => {
setFormState(prevState => ({
...prevState, prop_value: { ...prevState.prop_value, type: newType }
}))
}
function onPropAdd() {
setFormState(prevState => {
const id = prevState.entries[prevState.entries.length - 1] + 1
return {
entries: prevState.entries.concat([id]),
values: {
...prevState.values,
[id]: {
propKey: null,
type: FILTER_OPERATIONS.is,
clauses: []
}
}
}
})
}
function selectedFilterType() {
return formState.prop_value.type
}
function onPropDelete(id) {
setFormState(prevState => {
const entries = prevState.entries.filter((entry) => entry != id)
const values = {...prevState.values}
delete values[id]
function renderFilterInputs() {
return (
<div className="grid grid-cols-11 mt-6">
<div className="col-span-4">
<Combobox className="mr-2" fetchOptions={fetchPropKeyOptions()} singleOption={true} values={formState.prop_key ? [formState.prop_key] : []} onSelect={onPropKeySelect()} placeholder={'Property'} />
</div>
<div className="col-span-3 mx-2">
<FilterTypeSelector isDisabled={!formState.prop_key} forFilter={'prop_value'} onSelect={onFilterTypeSelect()} selectedType={selectedFilterType()} />
</div>
<div className="col-span-4">
<Combobox isDisabled={!formState.prop_key} fetchOptions={fetchPropValueOptions()} values={formState.prop_value.clauses} onSelect={onPropValueSelect()} placeholder={'Value'} />
</div>
</div>
)
return { entries, values }
})
}
function isDisabled() {
return !(formState.prop_key && formState.prop_value.clauses.length > 0)
return formState.entries.some((id) => !formState.values[id].propKey || formState.values[id].clauses.length == 0)
}
function shouldShowClear() {
@ -101,8 +113,11 @@ function PropFilterModal(props) {
}
function handleSubmit() {
const filterString = JSON.stringify({ [formState.prop_key.value]: toFilterQuery(formState.prop_value.type, formState.prop_value.clauses) })
selectFiltersAndCloseModal(filterString)
const formFilters = Object.fromEntries(
Object.entries(formState.values)
.map(([_, {propKey, type, clauses}]) => [propKey.value, toFilterQuery(type, clauses)])
)
selectFiltersAndCloseModal(JSON.stringify(formFilters))
}
function selectFiltersAndCloseModal(filterString) {
@ -139,7 +154,27 @@ function PropFilterModal(props) {
<div className="mt-4 border-b border-gray-300"></div>
<main className="modal__content">
<form className="flex flex-col" onSubmit={handleSubmit}>
{renderFilterInputs()}
{formState.entries.map((id) => (
<PropFilterRow
key={id}
id={id}
site={props.site}
query={query}
{...formState.values[id]}
showDelete={formState.entries.length > 1}
selectedPropKeys={selectedPropKeys}
onPropKeySelect={onPropKeySelect}
onPropValueSelect={onPropValueSelect}
onFilterTypeSelect={onFilterTypeSelect}
onPropDelete={onPropDelete}
/>
))}
<div className="mt-6">
<a className="underline text-indigo-500 text-sm cursor-pointer" onClick={onPropAdd}>
+ Add another
</a>
</div>
<div className="mt-6 flex items-center justify-start">
<button
@ -154,7 +189,7 @@ function PropFilterModal(props) {
<button
type="button"
className="ml-2 button px-4 flex bg-red-500 dark:bg-red-500 hover:bg-red-600 dark:hover:bg-red-700 items-center"
onClick={() => {selectFiltersAndCloseModal(null)}}
onClick={() => { selectFiltersAndCloseModal(null) }}
>
<svg className="w-4 h-4 mr-2" fill="none" stroke="currentColor" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path strokeLinecap="round" strokeLinejoin="round" strokeWidth="2" d="M19 7l-.867 12.142A2 2 0 0116.138 21H7.862a2 2 0 01-1.995-1.858L5 7m5 4v6m4-6v6m1-10V4a1 1 0 00-1-1h-4a1 1 0 00-1 1v3M4 7h16"></path></svg>
Remove filter
@ -167,4 +202,4 @@ function PropFilterModal(props) {
)
}
export default withRouter(PropFilterModal)
export default withRouter(PropFilterModal)

@ -0,0 +1,86 @@
import React from 'react'
import Combobox from '../../components/combobox'
import FilterTypeSelector from "../../components/filter-type-selector";
import { FILTER_OPERATIONS } from "../../util/filters";
import * as api from '../../api'
import { apiPath } from '../../util/url'
import { TrashIcon } from '@heroicons/react/20/solid'
function PropFilterRow({
id,
query,
site,
propKey,
type,
clauses,
showDelete,
selectedPropKeys,
onPropKeySelect,
onPropValueSelect,
onFilterTypeSelect,
onPropDelete
}) {
function fetchPropKeyOptions() {
return (input) => {
return api.get(apiPath(site, "/suggestions/prop_key"), query, { q: input.trim() })
}
}
function fetchPropValueOptions() {
return (input) => {
if (type === FILTER_OPERATIONS.contains) {
return Promise.resolve([])
}
const key = propKey?.value
const updatedQuery = { ...query, filters: { ...query.filters, props: { [key]: '!(none)' } } }
return api.get(apiPath(site, "/suggestions/prop_value"), updatedQuery, { q: input.trim() })
}
}
return (
<div className="grid grid-cols-12 mt-6">
<div className="col-span-4">
<Combobox
className="mr-2"
fetchOptions={fetchPropKeyOptions()}
singleOption
autoFocus
values={propKey ? [propKey] : []}
onSelect={(value) => onPropKeySelect(id, value)}
placeholder={'Property'}
disabledOptions={selectedPropKeys}
/>
</div>
<div className="col-span-3 mx-2">
<FilterTypeSelector
isDisabled={!propKey}
forFilter={'prop_value'}
onSelect={(value) => onFilterTypeSelect(id, value)}
selectedType={type}
/>
</div>
<div className="col-span-4">
<Combobox
isDisabled={!propKey}
fetchOptions={fetchPropValueOptions()}
values={clauses}
onSelect={(value) => onPropValueSelect(id, value)}
placeholder={'Value'}
freeChoice={type == FILTER_OPERATIONS.contains}
/>
</div>
{showDelete && (
<div className="col-span-1 flex flex-col justify-center">
<a className="ml-2 text-red-600 h-5 w-5 cursor-pointer" onClick={() => onPropDelete(id)}>
<TrashIcon />
</a>
</div>
)}
</div>
)
}
export default PropFilterRow

@ -1,7 +1,6 @@
import React, { useEffect, useState } from "react";
import { Link } from 'react-router-dom'
import { withRouter } from 'react-router-dom'
import Money from "../behaviours/money";
import Modal from './modal'
import * as api from '../../api'
@ -11,6 +10,18 @@ import { parseQuery } from '../../query'
import { specialTitleWhenGoalFilter } from "../behaviours/goal-conversions";
import { escapeFilterValue } from "../../util/filters"
/*global BUILD_EXTRA*/
/*global require*/
function maybeRequire() {
if (BUILD_EXTRA) {
return require('../../extra/money')
} else {
return { default: null }
}
}
const Money = maybeRequire().default
function PropsModal(props) {
const site = props.site
const query = parseQuery(props.location.search, site)
@ -81,11 +92,11 @@ function PropsModal(props) {
}
function renderBody() {
const hasRevenue = list.some((prop) => prop.total_revenue)
const hasRevenue = BUILD_EXTRA && list.some((prop) => prop.total_revenue)
return (
<>
<h1 className="text-xl font-bold dark:text-gray-100">{ specialTitleWhenGoalFilter(query, 'Custom Property Breakdown') }</h1>
<h1 className="text-xl font-bold dark:text-gray-100">{specialTitleWhenGoalFilter(query, 'Custom Property Breakdown')}</h1>
<div className="my-4 border-b border-gray-300"></div>
<main className="modal__content">

@ -3,7 +3,7 @@ import { withRouter } from 'react-router-dom'
import FilterTypeSelector from "../../components/filter-type-selector";
import Combobox from '../../components/combobox'
import { FILTER_GROUPS, parseQueryFilter, formatFilterGroup, formattedFilters, toFilterQuery, FILTER_TYPES } from '../../util/filters'
import { FILTER_GROUPS, parseQueryFilter, formatFilterGroup, formattedFilters, toFilterQuery, FILTER_OPERATIONS } from '../../util/filters'
import { parseQuery } from '../../query'
import * as api from '../../api'
import { apiPath, siteBasePath } from '../../util/url'
@ -32,7 +32,7 @@ class RegularFilterModal extends React.Component {
super(props)
const query = parseQuery(props.location.search, props.site)
const formState = getFormState(props.filterGroup, query)
this.handleKeydown = this.handleKeydown.bind(this)
this.state = { query, formState }
}
@ -92,7 +92,7 @@ class RegularFilterModal extends React.Component {
fetchOptions(filter) {
return (input) => {
const { query, formState } = this.state
if (formState[filter].type === FILTER_TYPES.contains) {return Promise.resolve([])}
if (formState[filter].type === FILTER_OPERATIONS.contains) {return Promise.resolve([])}
const formFilters = Object.fromEntries(
Object.entries(formState)

@ -20,8 +20,24 @@ class ModalTable extends React.Component {
.then((res) => this.setState({loading: false, list: res}))
}
showConversionRate() {
return !!this.state.query.filters.goal
}
showPercentage() {
return this.props.showPercentage && !this.showConversionRate()
}
label() {
return this.state.query.period === 'realtime' ? 'Current visitors' : 'Visitors'
if (this.state.query.period === 'realtime') {
return 'Current visitors'
}
if (this.showConversionRate()) {
return 'Conversions'
}
return 'Visitors'
}
renderTableItem(tableItem) {
@ -40,11 +56,10 @@ class ModalTable extends React.Component {
{tableItem.name}
</Link>
</td>
<td className="p-2 w-32 font-medium" align="right">
{numberFormatter(tableItem.visitors)}
{tableItem.percentage >= 0 &&
<span className="inline-block text-xs w-8 pl-1 text-right">({tableItem.percentage}%)</span> }
</td>
{this.showConversionRate() && <td className="p-2 w-32 font-medium" align="right">{numberFormatter(tableItem.total_visitors)}</td>}
<td className="p-2 w-32 font-medium" align="right">{numberFormatter(tableItem.visitors)}</td>
{this.showPercentage() && <td className="p-2 w-32 font-medium" align="right">{tableItem.percentage}</td>}
{this.showConversionRate() && <td className="p-2 w-32 font-medium" align="right">{numberFormatter(tableItem.conversion_rate)}%</td>}
</tr>
)
}
@ -66,19 +81,11 @@ class ModalTable extends React.Component {
<table className="w-max overflow-x-auto md:w-full table-striped table-fixed">
<thead>
<tr>
<th
className="p-2 w-48 lg:w-1/2 text-xs tracking-wide font-bold text-gray-500 dark:text-gray-400"
align="left"
>
{this.props.keyLabel}
</th>
<th
// eslint-disable-next-line max-len
className="p-2 w-32 lg:w-1/2 text-xs tracking-wide font-bold text-gray-500 dark:text-gray-400"
align="right"
>
{this.label()}
</th>
<th className="p-2 w-48 md:w-56 lg:w-1/3 text-xs tracking-wide font-bold text-gray-500 dark:text-gray-400" align="left">{this.props.keyLabel}</th>
{this.showConversionRate() && <th className="p-2 w-32 text-xs tracking-wide font-bold text-gray-500 dark:text-gray-400" align="right" >Total Visitors</th>}
<th className="p-2 w-32 text-xs tracking-wide font-bold text-gray-500 dark:text-gray-400" align="right">{this.label()}</th>
{this.showPercentage() && <th className="p-2 w-32 text-xs tracking-wide font-bold text-gray-500 dark:text-gray-400" align="right">%</th>}
{this.showConversionRate() && <th className="p-2 w-32 text-xs tracking-wide font-bold text-gray-500 dark:text-gray-400" align="right">CR</th>}
</tr>
</thead>
<tbody>

@ -306,8 +306,8 @@ export default function ListReport(props) {
const moreResultsAvailable = state.list.length >= MAX_ITEMS
const hideDetails = props.maybeHideDetails && !moreResultsAvailable
const showDetails = props.detailsLink && !state.loading && !hideDetails
return showDetails && <MoreLink className={'mt-2'} url={props.detailsLink} list={state.list} onClick={props.onClick}/>
const showDetails = props.detailsLink && !state.loading && !hideDetails
return showDetails && <MoreLink className={'mt-2'} url={props.detailsLink} list={state.list} />
}
return (

@ -1,6 +1,17 @@
import numberFormatter from "../../util/number-formatter"
import React from "react"
import Money from "../behaviours/money"
/*global BUILD_EXTRA*/
/*global require*/
function maybeRequire() {
if (BUILD_EXTRA) {
return require('../../extra/money')
} else {
return { default: null }
}
}
const Money = maybeRequire().default
export const VISITORS_METRIC = {
name: 'visitors',
@ -32,7 +43,7 @@ export function displayMetricValue(value, metric) {
} else if (metric === CR_METRIC) {
return `${value}%`
} else {
return <span tooltip={value}>{ numberFormatter(value) }</span>
return <span tooltip={value}>{numberFormatter(value)}</span>
}
}
@ -40,4 +51,4 @@ export function metricLabelFor(metric, query) {
if (metric.realtimeLabel && query.period === 'realtime') { return metric.realtimeLabel }
if (metric.goalFilterLabel && query.filters.goal) { return metric.goalFilterLabel }
return metric.label
}
}

@ -2,11 +2,13 @@ import React from 'react';
import SearchTerms from './search-terms'
import SourceList from './source-list'
import ReferrerList from './referrer-list'
import { isFilteringOnFixedValue } from '../../util/filters'
export default function Sources(props) {
if (props.query.filters.source === 'Google') {
return <SearchTerms {...props} />
} else if (props.query.filters.source) {
} else if (isFilteringOnFixedValue(props.query, 'source')) {
return <ReferrerList {...props} />
} else {
return <SourceList {...props} />

@ -7,6 +7,7 @@ import ListReport from '../reports/list'
import { VISITORS_METRIC, maybeWithCR } from '../reports/metrics';
import { Menu, Transition } from '@headlessui/react'
import { ChevronDownIcon } from '@heroicons/react/20/solid'
import { escapeFilterValue } from '../../util/filters'
import classNames from 'classnames'
const UTM_TAGS = {
@ -25,7 +26,7 @@ function AllSources(props) {
}
function getFilterFor(listItem) {
return { source: listItem['name'] }
return { source: escapeFilterValue(listItem['name']) }
}
function renderIcon(listItem) {
@ -60,7 +61,7 @@ function UTMSources(props) {
}
function getFilterFor(listItem) {
return { [props.tab]: listItem['name'] }
return { [props.tab]: escapeFilterValue(listItem['name']) }
}
return (

@ -1,3 +1,5 @@
const flags = JSON.parse(document.getElementById('stats-react-container').dataset.flags)
export const FILTER_GROUPS = {
'page': ['page', 'entry_page', 'exit_page'],
'source': ['source', 'referrer'],
@ -7,19 +9,23 @@ export const FILTER_GROUPS = {
'os': ['os', 'os_version'],
'utm': ['utm_medium', 'utm_source', 'utm_campaign', 'utm_term', 'utm_content'],
'goal': ['goal'],
'props': ['prop_key', 'prop_value']
'props': ['prop_key', 'prop_value'],
...(flags.hostname_filter ? { 'hostname': ['hostname'] } : {})
}
export const FILTER_TYPES = {
export const NO_CONTAINS_OPERATOR = new Set(['goal', 'screen'].concat(FILTER_GROUPS['location']))
export const FILTER_OPERATIONS = {
isNot: 'is not',
contains: 'contains',
is: 'is'
};
export const FILTER_PREFIXES = {
[FILTER_TYPES.isNot]: '!',
[FILTER_TYPES.contains]: '~',
[FILTER_TYPES.is]: ''
export const OPERATION_PREFIX = {
[FILTER_OPERATIONS.isNot]: '!',
[FILTER_OPERATIONS.contains]: '~',
[FILTER_OPERATIONS.is]: ''
};
export function supportsIsNot(filterName) {
@ -27,7 +33,7 @@ export function supportsIsNot(filterName) {
}
export function isFreeChoiceFilter(filterName) {
return FILTER_GROUPS['page'].concat(FILTER_GROUPS['utm']).includes(filterName)
return !NO_CONTAINS_OPERATOR.has(filterName)
}
// As of March 2023, Safari does not support negative lookbehind regexes. In case it throws an error, falls back to plain | matching. This means
@ -35,7 +41,7 @@ export function isFreeChoiceFilter(filterName) {
let NON_ESCAPED_PIPE_REGEX;
try {
NON_ESCAPED_PIPE_REGEX = new RegExp("(?<!\\\\)\\|", "g")
} catch(_e) {
} catch (_e) {
NON_ESCAPED_PIPE_REGEX = '|'
}
@ -46,56 +52,61 @@ export function escapeFilterValue(value) {
}
export function toFilterQuery(type, clauses) {
const prefix = FILTER_PREFIXES[type];
const prefix = OPERATION_PREFIX[type];
const result = clauses.map(clause => escapeFilterValue(clause.value.trim())).join('|')
return prefix + result;
}
export function parsePrefix(rawValue) {
const type = Object.keys(FILTER_PREFIXES)
.find(type => FILTER_PREFIXES[type] === rawValue[0]) || FILTER_TYPES.is;
const type = Object.keys(OPERATION_PREFIX)
.find(type => OPERATION_PREFIX[type] === rawValue[0]) || FILTER_OPERATIONS.is;
const value = type === FILTER_TYPES.is ? rawValue : rawValue.substring(1)
const value = type === FILTER_OPERATIONS.is ? rawValue : rawValue.substring(1)
const values = value
.split(NON_ESCAPED_PIPE_REGEX)
.filter((clause) => !!clause)
.map((val) => val.replaceAll(ESCAPED_PIPE, '|'))
return {type, values}
return { type, values }
}
export function parseQueryPropsFilter(query) {
return Object.entries(query.filters['props']).map(([key, propVal]) => {
const { type, values } = parsePrefix(propVal)
const clauses = values.map(val => { return { value: val, label: val } })
return { propKey: { label: key, value: key }, type, clauses }
})
}
export function parseQueryFilter(query, filter) {
if (filter === 'props') {
const rawValue = query.filters['props']
const [[_propKey, propVal]] = Object.entries(rawValue)
const {type, values} = parsePrefix(propVal)
const clauses = values.map(val => { return {value: val, label: val}})
return {type, clauses}
} else {
const {type, values} = parsePrefix(query.filters[filter] || '')
const { type, values } = parsePrefix(query.filters[filter] || '')
let labels = values
let labels = values
if (filter === 'country' && values.length > 0) {
const rawLabel = (new URLSearchParams(window.location.search)).get('country_labels') || ''
labels = rawLabel.split('|').filter(label => !!label)
}
if (filter === 'country' && values.length > 0) {
const rawLabel = (new URLSearchParams(window.location.search)).get('country_labels') || ''
labels = rawLabel.split('|').filter(label => !!label)
}
if (filter === 'region' && values.length > 0) {
const rawLabel = (new URLSearchParams(window.location.search)).get('region_labels') || ''
labels = rawLabel.split('|').filter(label => !!label)
}
if (filter === 'region' && values.length > 0) {
const rawLabel = (new URLSearchParams(window.location.search)).get('region_labels') || ''
labels = rawLabel.split('|').filter(label => !!label)
}
if (filter === 'city' && values.length > 0) {
const rawLabel = (new URLSearchParams(window.location.search)).get('city_labels') || ''
labels = rawLabel.split('|').filter(label => !!label)
}
if (filter === 'city' && values.length > 0) {
const rawLabel = (new URLSearchParams(window.location.search)).get('city_labels') || ''
labels = rawLabel.split('|').filter(label => !!label)
}
const clauses = values.map((value, index) => { return { value, label: labels[index] } })
const clauses = values.map((value, index) => { return {value, label: labels[index]}})
return { type, clauses }
}
return {type, clauses}
}
export function isFilteringOnFixedValue(query, filter) {
const { type, clauses } = parseQueryFilter(query, filter)
return type == FILTER_OPERATIONS.is && clauses.length == 1
}
export function formatFilterGroup(filterGroup) {
@ -145,6 +156,7 @@ export const formattedFilters = {
'region': 'Region',
'city': 'City',
'page': 'Page',
'hostname': 'Hostname',
'entry_page': 'Entry Page',
'exit_page': 'Exit Page'
'exit_page': 'Exit Page',
}

@ -5,6 +5,7 @@ export default (id) => ({
isOpen: false,
id: id,
focus: null,
selectionInProgress: false,
setFocus(f) {
this.focus = f;
},
@ -13,9 +14,14 @@ export default (id) => ({
this.setFocus(this.leastFocusableIndex())
}
},
trackSubmitValueChange() {
this.selectionInProgress = false;
},
open() {
this.initFocus()
this.isOpen = true
if (!this.isOpen) {
this.initFocus()
this.isOpen = true
}
},
suggestionsCount() {
return this.$refs.suggestions?.querySelectorAll('li').length
@ -61,7 +67,7 @@ export default (id) => ({
focusNext() {
const nextIndex = this.nextFocusableIndex()
if (!this.isOpen) this.open()
this.open()
this.setFocus(nextIndex)
this.scrollTo(nextIndex)
@ -69,7 +75,7 @@ export default (id) => ({
focusPrev() {
const prevIndex = this.prevFocusableIndex()
if (!this.isOpen) this.open()
this.open()
this.setFocus(prevIndex)
this.scrollTo(prevIndex)

@ -17,10 +17,4 @@ export default () => ({
this.open = false
focusAfter && focusAfter.focus()
},
onPanelClick(e) {
if (e.target.tagName === 'A') {
this.close()
}
}
})

@ -19,12 +19,30 @@ if (csrfToken && websocketUrl) {
})
}
}
let Uploaders = {}
Uploaders.S3 = function (entries, onViewError) {
entries.forEach(entry => {
let xhr = new XMLHttpRequest()
onViewError(() => xhr.abort())
xhr.onload = () => xhr.status === 200 ? entry.progress(100) : entry.error()
xhr.onerror = () => entry.error()
xhr.upload.addEventListener("progress", (event) => {
if (event.lengthComputable) {
let percent = Math.round((event.loaded / event.total) * 100)
if (percent < 100) { entry.progress(percent) }
}
})
let url = entry.meta.url
xhr.open("PUT", url, true)
xhr.send(entry.file)
})
}
let token = csrfToken.getAttribute("content")
let url = websocketUrl.getAttribute("content")
let liveUrl = (url === "") ? "/live" : new URL("/live", url).href;
let liveSocket = new LiveSocket(liveUrl, Socket, {
heartbeatIntervalMs: 10000,
params: { _csrf_token: token }, hooks: Hooks, dom: {
params: { _csrf_token: token }, hooks: Hooks, uploaders: Uploaders, dom: {
// for alpinejs integration
onBeforeElUpdated(from, to) {
if (from._x_dataStack) {

@ -2,11 +2,13 @@ BASE_URL=http://localhost:8000
SECURE_COOKIE=false
DATABASE_URL=postgres://postgres:postgres@127.0.0.1:5432/plausible_dev
CLICKHOUSE_DATABASE_URL=http://127.0.0.1:8123/plausible_events_db
CLICKHOUSE_MAX_BUFFER_SIZE_BYTES=1000000
SECRET_KEY_BASE=/njrhntbycvastyvtk1zycwfm981vpo/0xrvwjjvemdakc/vsvbrevlwsc6u8rcg
TOTP_VAULT_KEY=Q3BD4nddbkVJIPXgHuo5NthGKSIH0yesRfG05J88HIo=
ENVIRONMENT=dev
MAILER_ADAPTER=Bamboo.LocalAdapter
LOG_LEVEL=debug
SELFHOST=false
SELFHOST=true
DISABLE_CRON=true
ADMIN_USER_IDS=1
SHOW_CITIES=true
@ -18,3 +20,11 @@ GOOGLE_CLIENT_SECRET=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
PROMEX_DISABLED=false
SITE_DEFAULT_INGEST_THRESHOLD=1000000
S3_DISABLED=false
S3_ACCESS_KEY_ID=minioadmin
S3_SECRET_ACCESS_KEY=minioadmin
S3_REGION=us-east-1
S3_ENDPOINT=http://localhost:10000
S3_EXPORTS_BUCKET=dev-exports
S3_IMPORTS_BUCKET=dev-imports

@ -1,6 +1,7 @@
DATABASE_URL=postgres://postgres:postgres@127.0.0.1:5432/plausible_test?pool_size=40
CLICKHOUSE_DATABASE_URL=http://127.0.0.1:8123/plausible_test
SECRET_KEY_BASE=/njrhntbycvastyvtk1zycwfm981vpo/0xrvwjjvemdakc/vsvbrevlwsc6u8rcg
TOTP_VAULT_KEY=1Jah1HEOnCEnmBE+4/OgbJRraJIppPmYCNbZoFJboZs=
BASE_URL=http://localhost:8000
CRON_ENABLED=false
LOG_LEVEL=warning
@ -14,3 +15,11 @@ IP_GEOLOCATION_DB=test/priv/GeoLite2-City-Test.mmdb
SITE_DEFAULT_INGEST_THRESHOLD=1000000
GOOGLE_CLIENT_ID=fake_client_id
GOOGLE_CLIENT_SECRET=fake_client_secret
S3_DISABLED=false
S3_ACCESS_KEY_ID=minioadmin
S3_SECRET_ACCESS_KEY=minioadmin
S3_REGION=us-east-1
S3_ENDPOINT=http://localhost:10000
S3_EXPORTS_BUCKET=test-exports
S3_IMPORTS_BUCKET=test-imports

@ -20,7 +20,7 @@ config :esbuild,
version: "0.17.11",
default: [
args:
~w(js/app.js js/dashboard.js js/embed.host.js js/embed.content.js --bundle --target=es2017 --loader:.js=jsx --outdir=../priv/static/js),
~w(js/app.js js/dashboard.js js/embed.host.js js/embed.content.js --bundle --target=es2017 --loader:.js=jsx --outdir=../priv/static/js --define:BUILD_EXTRA=true),
cd: Path.expand("../assets", __DIR__),
env: %{"NODE_PATH" => Path.expand("../deps", __DIR__)}
]
@ -38,14 +38,14 @@ config :tailwind,
config :ua_inspector,
database_path: "priv/ua_inspector",
remote_release: "66d80de32fbb265941f4d7941fadc19097375097"
remote_release: "6.3.0"
config :ref_inspector,
database_path: "priv/ref_inspector"
config :plausible,
paddle_api: Plausible.Billing.PaddleApi,
google_api: Plausible.Google.Api
google_api: Plausible.Google.API
config :plausible,
# 30 minutes
@ -65,10 +65,7 @@ config :plausible, Plausible.Repo,
connect_timeout: 300_000,
handshake_timeout: 300_000
config :plausible,
sites_by_domain_cache_enabled: true,
sites_by_domain_cache_refresh_interval_max_jitter: :timer.seconds(5),
sites_by_domain_cache_refresh_interval: :timer.minutes(15)
config :plausible, Plausible.Cache, enabled: true
config :plausible, Plausible.Ingestion.Counters, enabled: true
@ -76,4 +73,8 @@ config :ex_cldr,
default_locale: "en",
default_backend: Plausible.Cldr
config :sentry,
enable_source_code_context: true,
root_source_code_path: [File.cwd!()]
import_config "#{config_env()}.exs"

@ -6,6 +6,14 @@ if config_env() in [:dev, :test] do
Envy.load(["config/.env.#{config_env()}"])
end
if config_env() == :small_dev do
Envy.load(["config/.env.dev"])
end
if config_env() == :small_test do
Envy.load(["config/.env.test"])
end
config_dir = System.get_env("CONFIG_DIR", "/run/secrets")
log_format =
@ -13,7 +21,7 @@ log_format =
log_level =
config_dir
|> get_var_from_path_or_env("LOG_LEVEL", "warn")
|> get_var_from_path_or_env("LOG_LEVEL", "warning")
|> String.to_existing_atom()
config :logger,
@ -132,11 +140,31 @@ ch_db_url =
|> get_var_from_path_or_env("CLICKHOUSE_FLUSH_INTERVAL_MS", "5000")
|> Integer.parse()
if get_var_from_path_or_env(config_dir, "CLICKHOUSE_MAX_BUFFER_SIZE") do
Logger.warning(
"CLICKHOUSE_MAX_BUFFER_SIZE is deprecated, please use CLICKHOUSE_MAX_BUFFER_SIZE_BYTES instead"
)
end
{ch_max_buffer_size, ""} =
config_dir
|> get_var_from_path_or_env("CLICKHOUSE_MAX_BUFFER_SIZE", "10000")
|> get_var_from_path_or_env("CLICKHOUSE_MAX_BUFFER_SIZE_BYTES", "100000")
|> Integer.parse()
# Can be generated with `Base.encode64(:crypto.strong_rand_bytes(32))` from
# iex shell or `openssl rand -base64 32` from command line.
totp_vault_key = get_var_from_path_or_env(config_dir, "TOTP_VAULT_KEY", nil)
case totp_vault_key do
nil ->
raise "TOTP_VAULT_KEY configuration option is required. See https://plausible.io/docs/self-hosting-configuration#server"
key ->
if byte_size(Base.decode64!(key)) != 32 do
raise "TOTP_VAULT_KEY must exactly 32 bytes long. See https://plausible.io/docs/self-hosting-configuration#server"
end
end
### Mandatory params End
build_metadata_raw = get_var_from_path_or_env(config_dir, "BUILD_METADATA", "{}")
@ -169,6 +197,8 @@ runtime_metadata = [
config :plausible, :runtime_metadata, runtime_metadata
config :plausible, Plausible.Auth.TOTP, vault_key: totp_vault_key
sentry_dsn = get_var_from_path_or_env(config_dir, "SENTRY_DSN")
honeycomb_api_key = get_var_from_path_or_env(config_dir, "HONEYCOMB_API_KEY")
honeycomb_dataset = get_var_from_path_or_env(config_dir, "HONEYCOMB_DATASET")
@ -199,11 +229,7 @@ ip_geolocation_db = get_var_from_path_or_env(config_dir, "IP_GEOLOCATION_DB", ge
geonames_source_file = get_var_from_path_or_env(config_dir, "GEONAMES_SOURCE_FILE")
maxmind_license_key = get_var_from_path_or_env(config_dir, "MAXMIND_LICENSE_KEY")
maxmind_edition = get_var_from_path_or_env(config_dir, "MAXMIND_EDITION", "GeoLite2-City")
maxmind_cache_dir = get_var_from_path_or_env(config_dir, "PERSISTENT_CACHE_DIR")
if System.get_env("DISABLE_AUTH") do
Logger.warning("DISABLE_AUTH env var is no longer supported")
end
persistent_cache_dir = get_var_from_path_or_env(config_dir, "PERSISTENT_CACHE_DIR")
enable_email_verification =
config_dir
@ -261,13 +287,16 @@ secure_cookie =
|> get_var_from_path_or_env("SECURE_COOKIE", if(is_selfhost, do: "false", else: "true"))
|> String.to_existing_atom()
license_key = get_var_from_path_or_env(config_dir, "LICENSE_KEY", "")
config :plausible,
environment: env,
mailer_email: mailer_email,
super_admin_user_ids: super_admin_user_ids,
is_selfhost: is_selfhost,
custom_script_name: custom_script_name,
log_failed_login_attempts: log_failed_login_attempts
log_failed_login_attempts: log_failed_login_attempts,
license_key: license_key
config :plausible, :selfhost,
enable_email_verification: enable_email_verification,
@ -306,21 +335,16 @@ else
database: get_var_from_path_or_env(config_dir, "DATABASE_NAME", "plausible")
end
included_environments = if sentry_dsn, do: ["prod", "staging", "dev"], else: []
sentry_app_version = runtime_metadata[:version] || app_version
config :sentry,
dsn: sentry_dsn,
environment_name: env,
included_environments: included_environments,
release: sentry_app_version,
tags: %{app_version: sentry_app_version},
enable_source_code_context: true,
root_source_code_path: [File.cwd!()],
client: Plausible.Sentry.Client,
send_max_attempts: 1,
filter: Plausible.SentryFilter,
before_send_event: {Plausible.SentryFilter, :before_send}
before_send: {Plausible.SentryFilter, :before_send}
config :plausible, :paddle,
vendor_auth_code: paddle_auth_code,
@ -330,8 +354,10 @@ config :plausible, :google,
client_id: google_cid,
client_secret: google_secret,
api_url: "https://www.googleapis.com",
reporting_api_url: "https://analyticsreporting.googleapis.com",
max_buffer_size: get_int_from_path_or_env(config_dir, "GOOGLE_MAX_BUFFER_SIZE", 10_000)
reporting_api_url: "https://analyticsreporting.googleapis.com"
config :plausible, :imported,
max_buffer_size: get_int_from_path_or_env(config_dir, "IMPORTED_MAX_BUFFER_SIZE", 10_000)
maybe_ch_ipv6 =
get_var_from_path_or_env(config_dir, "ECTO_CH_IPV6", "false")
@ -358,7 +384,8 @@ config :plausible, Plausible.ClickhouseRepo,
url: ch_db_url,
transport_opts: ch_transport_opts,
settings: [
readonly: 1
readonly: 1,
join_algorithm: "direct,parallel_hash"
]
config :plausible, Plausible.IngestRepo,
@ -368,7 +395,10 @@ config :plausible, Plausible.IngestRepo,
transport_opts: ch_transport_opts,
flush_interval_ms: ch_flush_interval_ms,
max_buffer_size: ch_max_buffer_size,
pool_size: ingest_pool_size
pool_size: ingest_pool_size,
settings: [
materialized_views_ignore_errors: 1
]
config :plausible, Plausible.AsyncInsertRepo,
queue_target: 500,
@ -378,7 +408,8 @@ config :plausible, Plausible.AsyncInsertRepo,
pool_size: 1,
settings: [
async_insert: 1,
wait_for_async_insert: 0
wait_for_async_insert: 0,
materialized_views_ignore_errors: 1
]
config :plausible, Plausible.ImportDeletionRepo,
@ -436,6 +467,20 @@ case mailer_adapter do
retries: get_var_from_path_or_env(config_dir, "SMTP_RETRIES") || 2,
no_mx_lookups: get_var_from_path_or_env(config_dir, "SMTP_MX_LOOKUPS_ENABLED") || true
"Bamboo.Mua" ->
config :plausible, Plausible.Mailer, adapter: Bamboo.Mua
if relay = get_var_from_path_or_env(config_dir, "SMTP_HOST_ADDR") do
port = get_int_from_path_or_env(config_dir, "SMTP_HOST_PORT", 25)
username = get_var_from_path_or_env(config_dir, "SMTP_USER_NAME")
password = get_var_from_path_or_env(config_dir, "SMTP_USER_PWD")
config :plausible, Plausible.Mailer,
auth: [username: username, password: password],
relay: relay,
port: port
end
"Bamboo.LocalAdapter" ->
config :plausible, Plausible.Mailer, adapter: Bamboo.LocalAdapter
@ -476,7 +521,9 @@ cloud_cron = [
# Daily at 15
{"0 15 * * *", Plausible.Workers.NotifyAnnualRenewal},
# Every midnight
{"0 0 * * *", Plausible.Workers.LockSites}
{"0 0 * * *", Plausible.Workers.LockSites},
# Daily at 8
{"0 8 * * *", Plausible.Workers.AcceptTrafficUntil}
]
crontab = if(is_selfhost, do: base_cron, else: base_cron ++ cloud_cron)
@ -489,8 +536,11 @@ base_queues = [
check_stats_emails: 1,
site_setup_emails: 1,
clean_invitations: 1,
google_analytics_imports: 1,
domain_change_transition: 1
analytics_imports: 1,
domain_change_transition: 1,
check_accept_traffic_until: 1,
# NOTE: maybe move s3_csv_export to cloud_queues?
s3_csv_export: 1
]
cloud_queues = [
@ -514,8 +564,7 @@ cond do
{Oban.Plugins.Pruner, max_age: thirty_days_in_seconds},
{Oban.Plugins.Cron, crontab: if(cron_enabled, do: crontab, else: [])},
# Rescue orphaned jobs after 2 hours
{Oban.Plugins.Lifeline, rescue_after: :timer.minutes(120)},
{Oban.Plugins.Stager, interval: :timer.seconds(5)}
{Oban.Plugins.Lifeline, rescue_after: :timer.minutes(120)}
],
queues: if(cron_enabled, do: queues, else: []),
peer: if(cron_enabled, do: Oban.Peers.Postgres, else: false)
@ -546,35 +595,34 @@ config :ref_inspector,
config :ua_inspector,
init: {Plausible.Release, :configure_ua_inspector}
config :hammer,
backend: {Hammer.Backend.ETS, [expiry_ms: 60_000 * 60 * 4, cleanup_interval_ms: 60_000 * 10]}
config :kaffy,
otp_app: :plausible,
ecto_repo: Plausible.Repo,
router: PlausibleWeb.Router,
admin_title: "Plausible Admin",
resources: [
auth: [
resources: [
user: [schema: Plausible.Auth.User, admin: Plausible.Auth.UserAdmin],
api_key: [schema: Plausible.Auth.ApiKey, admin: Plausible.Auth.ApiKeyAdmin]
]
],
sites: [
resources: [
site: [schema: Plausible.Site, admin: Plausible.SiteAdmin]
]
],
billing: [
resources: [
enterprise_plan: [
schema: Plausible.Billing.EnterprisePlan,
admin: Plausible.Billing.EnterprisePlanAdmin
if config_env() in [:dev, :staging, :prod] do
config :kaffy,
otp_app: :plausible,
ecto_repo: Plausible.Repo,
router: PlausibleWeb.Router,
admin_title: "Plausible Admin",
resources: [
auth: [
resources: [
user: [schema: Plausible.Auth.User, admin: Plausible.Auth.UserAdmin],
api_key: [schema: Plausible.Auth.ApiKey, admin: Plausible.Auth.ApiKeyAdmin]
]
],
sites: [
resources: [
site: [schema: Plausible.Site, admin: Plausible.SiteAdmin]
]
],
billing: [
resources: [
enterprise_plan: [
schema: Plausible.Billing.EnterprisePlan,
admin: Plausible.Billing.EnterprisePlanAdmin
]
]
]
]
]
end
geo_opts =
cond do
@ -582,7 +630,7 @@ geo_opts =
[
license_key: maxmind_license_key,
edition: maxmind_edition,
cache_dir: maxmind_cache_dir,
cache_dir: persistent_cache_dir,
async: true
]
@ -634,9 +682,7 @@ else
traces_exporter: :none
end
config :tzdata,
:data_dir,
get_var_from_path_or_env(config_dir, "STORAGE_DIR", Application.app_dir(:tzdata, "priv"))
config :tzdata, :data_dir, Path.join(persistent_cache_dir || System.tmp_dir!(), "tzdata_data")
promex_disabled? =
config_dir
@ -663,3 +709,73 @@ if not is_selfhost do
config :plausible, Plausible.Site, default_ingest_threshold: site_default_ingest_threshold
end
s3_disabled? =
config_dir
|> get_var_from_path_or_env("S3_DISABLED", "true")
|> String.to_existing_atom()
unless s3_disabled? do
s3_env = [
%{
name: "S3_ACCESS_KEY_ID",
example: "AKIAIOSFODNN7EXAMPLE"
},
%{
name: "S3_SECRET_ACCESS_KEY",
example: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
},
%{
name: "S3_REGION",
example: "us-east-1"
},
%{
name: "S3_ENDPOINT",
example: "https://<ACCOUNT_ID>.r2.cloudflarestorage.com"
},
%{
name: "S3_EXPORTS_BUCKET",
example: "my-csv-exports-bucket"
},
%{
name: "S3_IMPORTS_BUCKET",
example: "my-csv-imports-bucket"
}
]
s3_env =
Enum.map(s3_env, fn var ->
Map.put(var, :value, get_var_from_path_or_env(config_dir, var.name))
end)
s3_missing_env = Enum.filter(s3_env, &is_nil(&1.value))
unless s3_missing_env == [] do
raise ArgumentError, """
Missing S3 configuration. Please set #{s3_missing_env |> Enum.map(& &1.name) |> Enum.join(", ")} environment variable(s):
#{s3_missing_env |> Enum.map(fn %{name: name, example: example} -> "\t#{name}=#{example}" end) |> Enum.join("\n")}
"""
end
s3_env_value = fn name ->
s3_env |> Enum.find(&(&1.name == name)) |> Map.fetch!(:value)
end
config :ex_aws,
http_client: Plausible.S3.Client,
access_key_id: s3_env_value.("S3_ACCESS_KEY_ID"),
secret_access_key: s3_env_value.("S3_SECRET_ACCESS_KEY"),
region: s3_env_value.("S3_REGION")
%URI{scheme: s3_scheme, host: s3_host, port: s3_port} = URI.parse(s3_env_value.("S3_ENDPOINT"))
config :ex_aws, :s3,
scheme: s3_scheme <> "://",
host: s3_host,
port: s3_port
config :plausible, Plausible.S3,
exports_bucket: s3_env_value.("S3_EXPORTS_BUCKET"),
imports_bucket: s3_env_value.("S3_IMPORTS_BUCKET")
end

@ -0,0 +1,11 @@
import Config
import_config "prod.exs"
config :esbuild,
default: [
args:
~w(js/app.js js/dashboard.js js/embed.host.js js/embed.content.js --bundle --target=es2017 --loader:.js=jsx --outdir=../priv/static/js --define:BUILD_EXTRA=false),
cd: Path.expand("../assets", __DIR__),
env: %{"NODE_PATH" => Path.expand("../deps", __DIR__)}
]

@ -0,0 +1,11 @@
import Config
import_config "dev.exs"
config :esbuild,
default: [
args:
~w(js/app.js js/dashboard.js js/embed.host.js js/embed.content.js --bundle --target=es2017 --loader:.js=jsx --outdir=../priv/static/js --define:BUILD_EXTRA=false),
cd: Path.expand("../assets", __DIR__),
env: %{"NODE_PATH" => Path.expand("../deps", __DIR__)}
]

@ -0,0 +1,3 @@
import Config
import_config "test.exs"

@ -16,15 +16,18 @@ config :plausible, Plausible.Mailer, adapter: Bamboo.TestAdapter
config :plausible,
paddle_api: Plausible.PaddleApi.Mock,
google_api: Plausible.Google.Api.Mock
google_api: Plausible.Google.API.Mock
config :bamboo, :refute_timeout, 10
config :plausible,
session_timeout: 0,
http_impl: Plausible.HTTPClient.Mock,
sites_by_domain_cache_enabled: false
http_impl: Plausible.HTTPClient.Mock
config :plausible, Plausible.Cache, enabled: false
config :ex_money, api_module: Plausible.ExchangeRateMock
config :plausible, Plausible.Ingestion.Counters, enabled: false
config :plausible, Oban, testing: :manual

@ -0,0 +1,5 @@
Plausible copyright notice
This software is Copyright 2024 Plausible Insights OÜ
This repository is provided for informational purposes only, and no rights to use, distribute or otherwise exploit this software are granted to you. Our other repositories include software that is offered under the Affero GPL, but this repository is not. If you wish to discuss a license, or our hosted service that uses the software, please contact us at hello@plausible.io

@ -0,0 +1,42 @@
defmodule Plausible.License do
@moduledoc """
This module ensures that you cannot run Plausible Analytics Enterprise Edition without a valid license key.
The software contained within the ee/ and assets/js/dashboard/ee directories are Copyright © Plausible Insights OÜ.
We have made this code available solely for informational and transparency purposes. No rights are granted to use,
distribute, or exploit this software in any form.
Any attempt to disable or modify the behavior of this module will be considered a violation of copyright.
If you wish to use the Plausible Analytics Enterprise Edition for your own requirements, please contact us
at hello@plausible.io to discuss obtaining a license.
"""
require Logger
if Mix.env() == :prod do
def ensure_valid_license do
if has_valid_license?() do
:ok
else
Logger.error(
"Invalid or no license key provided for Plausible Enterprise Edition. Please contact hello@plausible.io to acquire a license."
)
Logger.error("Shutting down")
System.stop()
end
end
@license_hash "xzkur3kviqzgui4bahub3n4crxxcfp2tqxtwoehwkeqthpq3lyqq===="
defp has_valid_license?() do
hash =
:crypto.hash(:sha256, Application.fetch_env!(:plausible, :license_key))
|> Base.encode32(case: :lower)
hash == @license_hash
end
else
def ensure_valid_license do
:ok
end
end
end

@ -0,0 +1,22 @@
defmodule Plausible.Goal.Revenue do
@moduledoc """
Currency specific functions for revenue goals
"""
def revenue?(%Plausible.Goal{currency: currency}) do
!!currency
end
def valid_currencies() do
Ecto.Enum.dump_values(Plausible.Goal, :currency)
end
def currency_options() do
options =
for code <- valid_currencies() do
{code, "#{code} - #{Cldr.Currency.display_name!(code)}"}
end
options
end
end

@ -0,0 +1,37 @@
defmodule Plausible.Ingestion.Event.Revenue do
@moduledoc """
Revenue specific functions for the ingestion scope
"""
def get_revenue_attrs(
%Plausible.Ingestion.Event{request: %{revenue_source: %Money{} = revenue_source}} = event
) do
matching_goal =
Enum.find(event.site.revenue_goals, &(&1.event_name == event.clickhouse_event_attrs.name))
cond do
is_nil(matching_goal) ->
%{}
matching_goal.currency == revenue_source.currency ->
%{
revenue_source_amount: Money.to_decimal(revenue_source),
revenue_source_currency: to_string(revenue_source.currency),
revenue_reporting_amount: Money.to_decimal(revenue_source),
revenue_reporting_currency: to_string(revenue_source.currency)
}
matching_goal.currency != revenue_source.currency ->
converted = Money.to_currency!(revenue_source, matching_goal.currency)
%{
revenue_source_amount: Money.to_decimal(revenue_source),
revenue_source_currency: to_string(revenue_source.currency),
revenue_reporting_amount: Money.to_decimal(converted),
revenue_reporting_currency: to_string(converted.currency)
}
end
end
def get_revenue_attrs(_event), do: %{}
end

@ -0,0 +1,35 @@
defmodule Plausible.Ingestion.Request.Revenue do
@moduledoc """
Revenue specific functions for the ingestion scope
"""
def put_revenue_source(%Ecto.Changeset{} = changeset, %{} = request_body) do
with revenue_source <- request_body["revenue"] || request_body["$"],
%{"amount" => _, "currency" => _} = revenue_source <-
Plausible.Helpers.JSON.decode_or_fallback(revenue_source) do
parse_revenue_source(changeset, revenue_source)
else
_any -> changeset
end
end
@valid_currencies Plausible.Goal.Revenue.valid_currencies()
defp parse_revenue_source(changeset, %{"amount" => amount, "currency" => currency}) do
with true <- currency in @valid_currencies,
{%Decimal{} = amount, _rest} <- parse_decimal(amount),
%Money{} = amount <- Money.new(currency, amount) do
Ecto.Changeset.put_change(changeset, :revenue_source, amount)
else
_any -> changeset
end
end
defp parse_decimal(value) do
case value do
value when is_binary(value) -> Decimal.parse(value)
value when is_float(value) -> {Decimal.from_float(value), nil}
value when is_integer(value) -> {Decimal.new(value), nil}
_any -> :error
end
end
end

@ -0,0 +1,93 @@
defmodule Plausible.Stats.Goal.Revenue do
@moduledoc """
Revenue specific functions for the stats scope
"""
import Ecto.Query
@revenue_metrics [:average_revenue, :total_revenue]
def revenue_metrics() do
@revenue_metrics
end
def total_revenue_query() do
dynamic(
[e],
fragment("toDecimal64(sum(?) * any(_sample_factor), 3)", e.revenue_reporting_amount)
)
end
def average_revenue_query() do
dynamic(
[e],
fragment("toDecimal64(avg(?) * any(_sample_factor), 3)", e.revenue_reporting_amount)
)
end
@spec get_revenue_tracking_currency(Plausible.Site.t(), Plausible.Stats.Query.t(), [atom()]) ::
{atom() | nil, [atom()]}
@doc """
Returns the common currency for the goal filters in a query. If there are no
goal filters, multiple currencies or the site owner does not have access to
revenue goals, `nil` is returned and revenue metrics are dropped.
Aggregating revenue data works only for same currency goals. If the query is
filtered by goals with different currencies, for example, one USD and other
EUR, revenue metrics are dropped.
"""
def get_revenue_tracking_currency(site, query, metrics) do
goal_filters =
case query.filters do
%{"event:goal" => {:is, {_, goal_name}}} -> [goal_name]
%{"event:goal" => {:member, list}} -> Enum.map(list, fn {_, goal_name} -> goal_name end)
_any -> []
end
requested_revenue_metrics? = Enum.any?(metrics, &(&1 in @revenue_metrics))
filtering_by_goal? = Enum.any?(goal_filters)
revenue_goals_available? = fn ->
site = Plausible.Repo.preload(site, :owner)
Plausible.Billing.Feature.RevenueGoals.check_availability(site.owner) == :ok
end
if requested_revenue_metrics? && filtering_by_goal? && revenue_goals_available?.() do
revenue_goals_currencies =
Plausible.Repo.all(
from rg in Ecto.assoc(site, :revenue_goals),
where: rg.event_name in ^goal_filters,
select: rg.currency,
distinct: true
)
if length(revenue_goals_currencies) == 1,
do: {List.first(revenue_goals_currencies), metrics},
else: {nil, metrics -- @revenue_metrics}
else
{nil, metrics -- @revenue_metrics}
end
end
def cast_revenue_metrics_to_money([%{goal: _goal} | _rest] = results, revenue_goals)
when is_list(revenue_goals) do
for result <- results do
if matching_goal = Enum.find(revenue_goals, &(&1.event_name == result.goal)) do
cast_revenue_metrics_to_money(result, matching_goal.currency)
else
result
end
end
end
def cast_revenue_metrics_to_money(results, currency) when is_map(results) do
for {metric, value} <- results, into: %{} do
if metric in @revenue_metrics && currency do
{metric, Money.new!(value || 0, currency)}
else
{metric, value}
end
end
end
def cast_revenue_metrics_to_money(results, _), do: results
end

@ -0,0 +1,41 @@
defmodule Plausible.Stats.Sampling do
@moduledoc """
Sampling related functions
"""
@default_sample_threshold 20_000_000
import Ecto.Query
@spec add_query_hint(Ecto.Query.t(), Plausible.Stats.Query.t()) :: Ecto.Query.t()
def add_query_hint(%Ecto.Query{} = db_query, %Plausible.Stats.Query{} = query) do
case query.sample_threshold do
:infinite ->
db_query
threshold ->
add_query_hint(db_query, threshold)
end
end
@spec add_query_hint(Ecto.Query.t(), pos_integer()) :: Ecto.Query.t()
def add_query_hint(%Ecto.Query{} = query, threshold) when is_integer(threshold) do
from(x in query, hints: unsafe_fragment(^"SAMPLE #{threshold}"))
end
@spec add_query_hint(Ecto.Query.t()) :: Ecto.Query.t()
def add_query_hint(%Ecto.Query{} = query) do
add_query_hint(query, @default_sample_threshold)
end
@spec put_threshold(Plausible.Stats.Query.t(), map()) :: Plausible.Stats.Query.t()
def put_threshold(query, params) do
sample_threshold =
case params["sample_threshold"] do
nil -> @default_sample_threshold
"infinite" -> :infinite
value -> String.to_integer(value)
end
Map.put(query, :sample_threshold, sample_threshold)
end
end

@ -1,7 +1,7 @@
defmodule PlausibleWeb.Api.ExternalSitesController do
use PlausibleWeb, :controller
use Plausible.Repo
use Plug.ErrorHandler
use PlausibleWeb.Plugs.ErrorHandler
alias Plausible.Sites
alias Plausible.Goals
alias PlausibleWeb.Api.Helpers, as: H
@ -13,7 +13,7 @@ defmodule PlausibleWeb.Api.ExternalSitesController do
{:ok, %{site: site}} ->
json(conn, site)
{:error, :limit, limit, _} ->
{:error, {:over_limit, limit}} ->
conn
|> put_status(402)
|> json(%{
@ -161,8 +161,4 @@ defmodule PlausibleWeb.Api.ExternalSitesController do
error_msg = Atom.to_string(field) <> ": " <> msg
%{"error" => error_msg}
end
def handle_errors(conn, %{kind: kind, reason: reason}) do
json(conn, %{error: Exception.format_banner(kind, reason)})
end
end

@ -0,0 +1,27 @@
defmodule PlausibleWeb.Controllers.API.Revenue do
@moduledoc """
Revenue specific functions for the API scope
"""
@revenue_metrics Plausible.Stats.Goal.Revenue.revenue_metrics()
def format_revenue_metric({metric, value}) do
if metric in @revenue_metrics do
{metric, format_money(value)}
else
{metric, value}
end
end
def format_money(value) do
case value do
%Money{} ->
%{
short: Money.to_string!(value, format: :short, fractional_digits: 1),
long: Money.to_string!(value)
}
_any ->
value
end
end
end

@ -0,0 +1,31 @@
defmodule PlausibleWeb.Dogfood do
@moduledoc """
Plausible tracking itself functions
"""
@doc """
Temporary override to do more testing of the new ingest.plausible.io endpoint for accepting events. In staging and locally
will fall back to staging.plausible.io/api/event and localhost:8000/api/event respectively.
"""
def api_destination() do
if Application.get_env(:plausible, :environment) == "prod" do
"https://ingest.plausible.io/api/event"
end
end
def script_url() do
if Application.get_env(:plausible, :environment) in ["prod", "staging"] do
"#{PlausibleWeb.Endpoint.url()}/js/script.manual.pageview-props.tagged-events.js"
else
"#{PlausibleWeb.Endpoint.url()}/js/script.local.manual.pageview-props.tagged-events.js"
end
end
def domain(conn) do
cond do
Application.get_env(:plausible, :is_selfhost) -> "ee.plausible.io"
conn.assigns[:embedded] -> "embed." <> PlausibleWeb.Endpoint.host()
true -> PlausibleWeb.Endpoint.host()
end
end
end

@ -2,7 +2,7 @@ defmodule PlausibleWeb.Live.FunnelSettings do
@moduledoc """
LiveView allowing listing, creating and deleting funnels.
"""
use Phoenix.LiveView
use PlausibleWeb, :live_view
use Phoenix.HTML
use Plausible.Funnel
@ -41,7 +41,7 @@ defmodule PlausibleWeb.Live.FunnelSettings do
def render(assigns) do
~H"""
<div id="funnel-settings-main">
<.live_component id="embedded_liveview_flash" module={PlausibleWeb.Live.Flash} flash={@flash} />
<.flash_messages flash={@flash} />
<%= if @add_funnel? do %>
<%= live_render(
@socket,
@ -101,8 +101,7 @@ defmodule PlausibleWeb.Live.FunnelSettings do
id = String.to_integer(id)
:ok = Funnels.delete(site, id)
socket = put_flash(socket, :success, "Funnel deleted successfully")
Process.send_after(self(), :clear_flash, 5000)
socket = put_live_flash(socket, :success, "Funnel deleted successfully")
{:noreply,
assign(socket,
@ -112,8 +111,7 @@ defmodule PlausibleWeb.Live.FunnelSettings do
end
def handle_info({:funnel_saved, funnel}, socket) do
socket = put_flash(socket, :success, "Funnel saved successfully")
Process.send_after(self(), :clear_flash, 5000)
socket = put_live_flash(socket, :success, "Funnel saved successfully")
{:noreply,
assign(socket,
@ -126,8 +124,4 @@ defmodule PlausibleWeb.Live.FunnelSettings do
def handle_info(:cancel_add_funnel, socket) do
{:noreply, assign(socket, add_funnel?: false)}
end
def handle_info(:clear_flash, socket) do
{:noreply, clear_flash(socket)}
end
end

@ -5,7 +5,7 @@ defmodule PlausibleWeb.Live.FunnelSettings.Form do
to allow building searchable funnel definitions out of list of goals available.
"""
use Phoenix.LiveView
use PlausibleWeb, :live_view
use Plausible.Funnel
import PlausibleWeb.Live.Components.Form
@ -81,9 +81,14 @@ defmodule PlausibleWeb.Live.FunnelSettings.Form do
<div :for={step_idx <- @step_ids} class="flex mb-3 mt-3">
<div class="w-2/5 flex-1">
<.live_component
submit_name="funnel[steps][][goal_id]"
submit_name={"funnel[steps][#{step_idx}][goal_id]"}
module={PlausibleWeb.Live.Components.ComboBox}
suggest_fun={&PlausibleWeb.Live.Components.ComboBox.StaticSearch.suggest/2}
on_selection_made={
fn value, by_id ->
send(self(), {:selection_made, %{submit_value: value, by: by_id}})
end
}
id={"step-#{step_idx}"}
options={reject_already_selected("step-#{step_idx}", @goals, @selections_made)}
/>
@ -223,11 +228,19 @@ defmodule PlausibleWeb.Live.FunnelSettings.Form do
end
def handle_event("validate", %{"funnel" => params}, socket) do
steps_from_assigns =
socket.assigns.step_ids
|> Enum.reduce([], fn step_id, acc ->
goal = Map.get(socket.assigns.selections_made, "step-#{step_id}")
if goal, do: [%{"goal_id" => goal.id} | acc], else: acc
end)
|> Enum.reverse()
changeset =
socket.assigns.site
|> Plausible.Funnels.create_changeset(
params["name"],
params["steps"] || []
steps_from_assigns
)
|> Map.put(:action, :validate)
@ -235,11 +248,13 @@ defmodule PlausibleWeb.Live.FunnelSettings.Form do
end
def handle_event("save", %{"funnel" => params}, %{assigns: %{site: site}} = socket) do
case Plausible.Funnels.create(site, params["name"], params["steps"]) do
steps = Enum.map(params["steps"], fn {_idx, payload} -> payload end)
case Plausible.Funnels.create(site, params["name"], steps) do
{:ok, funnel} ->
send(
socket.parent_pid,
{:funnel_saved, Map.put(funnel, :steps_count, length(params["steps"]))}
{:funnel_saved, Map.put(funnel, :steps_count, length(steps))}
)
{:noreply, socket}

@ -0,0 +1,38 @@
{
"kind": "analyticsData#batchRunReports",
"reports": [
{
"dimensionHeaders": [
{
"name": "date"
}
],
"kind": "analyticsData#runReport",
"metadata": {
"currencyCode": "USD",
"timeZone": "Europe/Warsaw"
},
"metricHeaders": [
{
"name": "screenPageViews",
"type": "TYPE_INTEGER"
}
],
"rowCount": 3,
"rows": [
{
"dimensionValues": [
{
"value": "20240302"
}
],
"metricValues": [
{
"value": "13"
}
]
}
]
}
]
}

@ -0,0 +1,12 @@
{
"account": "accounts/28425555",
"createTime": "2024-02-22T10:50:15.462Z",
"currencyCode": "USD",
"displayName": "account.one - GA4",
"name": "properties/428685444",
"parent": "accounts/28425555",
"propertyType": "PROPERTY_TYPE_ORDINARY",
"serviceLevel": "GOOGLE_ANALYTICS_STANDARD",
"timeZone": "Europe/Warsaw",
"updateTime": "2024-02-22T10:50:15.462Z"
}

@ -0,0 +1,41 @@
{
"accountSummaries": [
{
"account": "accounts/28425178",
"displayName": "account.one",
"name": "accountSummaries/28425178",
"propertySummaries": [
{
"displayName": "account.one - GA4",
"parent": "accounts/28425178",
"property": "properties/428685906",
"propertyType": "PROPERTY_TYPE_ORDINARY"
}
]
},
{
"account": "accounts/45336102",
"displayName": "account.two",
"name": "accountSummaries/45336102"
},
{
"account": "accounts/54516992",
"displayName": "Demo Account",
"name": "accountSummaries/54516992",
"propertySummaries": [
{
"displayName": "GA4 - Flood-It!",
"parent": "accounts/54516992",
"property": "properties/153293282",
"propertyType": "PROPERTY_TYPE_ORDINARY"
},
{
"displayName": "GA4 - Google Merch Shop",
"parent": "accounts/54516992",
"property": "properties/213025502",
"propertyType": "PROPERTY_TYPE_ORDINARY"
}
]
}
]
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More

Loading…
Cancel
Save