Commit Graph

1245 Commits (db5488efa677563cc3fce874f72ccb4e7c939708)

Author SHA1 Message Date
Vini Brasil eb6baa92a3
Show imported icon only when querying imported ranges (#2892)
This commit updates the imported 'G' icon, so that it appears only when
querying or comparing a range with imported data. By hiding the icon in
all other cases, users will see it only when they can actually click on
it.
4 years ago
RobertJoonas 7d935b79bf
Treat page filter as entry page filter only for bounce rate (#2874)
* add entry page filter condition to bounce rate

* Fix select_merge with dynamic

* fix tests and add one test

* generalize page_filter_condition function

* use dynamic_filter_condition for session filters too

* disable views_per_visit with a page filter

* update changelog

* disable credo for complex function

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
4 years ago
RobertJoonas caeff41932
Prop filter modal (#2841)
* Add PropFilterModal (only UI)

* small variable refactor

* allow selecting prop value filter type

* allow selecting only one prop_key

* allow selecting many prop_values only when prop_key selected

* handle submitting filter

* get applied filters from query + remove option

* change prop filter label format

* support member and not_member filter types for pageview props

* show (none) value in filter suggestions

* refactor zip_results/4 and remove unused code

* fix displaying (none) values in goals section prop breakdown

* remove unnecessary functionality

* fix bug: returning prop names for goal :member filter

* fix bug: submitting regular filter modal with Enter key

* bugfix: disallow opening prop filter modal when feature flag disabled

* mix format

* break selected combobox values into multiple lines

* fix useEffect behavior for focusing on prop_key field

* support submitting prop filter with Enter key

* refactor getFormState in PropFilterModal

* separate fetchPropKey and fetchPropValue functions

* Allow querying props for pageview goals

* Make the internal props API only return a list of props (not map)
* Separate function for fetching all props in Stats API goal breakdown (this returns a map as before)

* ditch state for keeping search bar visible

* group by event_name in db query
4 years ago
Vini Brasil 8dc8423319
Prepare Comparisons for release (#2873)
This pull request removes the comparisons feature flag, making it visible to all users. It also removes percentages from top stats in default view, leaving those for the comparison view only.
4 years ago
Vini Brasil 8ec3dd402f
Prevent overwriting with_imported when comparing (#2871)
This commit fixes a bug where comparisons would not work with imported data. This is because comparisons copies the source query and modifies the dates only, where it should reevaluate if there is imported data for the new comparison date range.

Closes #2870

Co-authored-by: Adam <hq@mtod.org>
4 years ago
hq1 71ef0bd043
Clean up after V2 migration (#2868)
* Clean up after V2 migration

This PR removes all the leftovers and alternative code
branching after v2 migration.

The self-hosted release is being drafted at:

https://github.com/plausible/hosting/issues/68

Refs:
  - https://github.com/plausible/analytics/pull/2865
  - https://github.com/plausible/analytics/pull/2825
  - https://github.com/plausible/analytics/pull/2780

* !fixup
4 years ago
ruslandoga adcce15632
Make self-hosted data migration easier (#2865)
* default to v2

* allow N defaults in data migration prompt and custom messages

* join domains lookup

* remove duplicate test runs from ci (both are v2)
4 years ago
Marko Saric 586bbeb8ac
Adding a note to contact us for more API calls (#2858)
this should make it clearer to people that they can get more API capacity when they encounter the "Too many API requests" error
4 years ago
Vini Brasil 3d656ae95b
Match day of the week feature for comparisons (#2822)
* Add support for `match_day_of_week?` back-end option

* Add match day of week input to comparison input

* Default match day of the week to true
4 years ago
hq1 825a754976
Make ingest threshold configurable (#2845)
* Make ingest threshold configurable

* Credo
4 years ago
hq1 b9c2110472
V2 migration tweaks for self hosted release (#2825)
* Get rid of PASS_V2_SCHEMA_MIGRATION

* Use in-memory domain lookup + regular table settings

* Remove faulty date arithmetic + prev part calculation

* Set V2_MIGRATION_DONE in Mix.env == :dev

* Mute credo
4 years ago
Vini Brasil 9f036d3915
Show detailed comparison dates in top stats (#2804)
* Pass comparison_query as argument to top stats functions

* Add comparison values and dates to top stats API

* Display comparison dates and values in top stats

* Rename function to renderPercentageComparison

* Create component to do conditional rendering

* DRY date range formatting function

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
4 years ago
hq1 c04e9286dd
Associate goals with sites, not domains (#2828)
* Revert "Rephrase error message"

This reverts commit f624443a96.

* Revert "Temporarily disable goal creation"

This reverts commit a091635b9d.

* Update ecto schema

* Make sure goal operations are per site

* Update tests

* Split postgres migrations
4 years ago
hq1 3cb089eab4
Migrate and freeze goals creation (#2833)
* Full migration (to be submitted separately)

* Do not remove `Goal.domain` just yet

* Do not make Goal.site not nullable just yet

* Temporarily disable goal creation

* Rephrase error message

* Add down migration
4 years ago
hq1 154ce3a44c
Split clickhouse repos - making the main one read only (#2826)
* Split clickhouse pools into readonly/import deletions

* Remove CRM site transfers

* Initialize ImportDeletionRepo

* Put ImportDeletionRepo to use
4 years ago
Uku Taht 71e4d228c0
Escape pipes in goal names (#2814)
* Escape pipes in goal names

* Apply suggestions from code review

Use ~S sigil for better readability

Co-authored-by: Vini Brasil <vini@hey.com>

---------

Co-authored-by: Vini Brasil <vini@hey.com>
4 years ago
hq1 406ea6f0cd
Render an error on site creation with `domain=domain_changed_from` (#2821) 4 years ago
hq1 ffc96205f6
Make general settings consistent with the rest (#2819) 4 years ago
hq1 1d01328287
Allow domain change (#2803)
* Migration (PR: https://github.com/plausible/analytics/pull/2802)

* Implement Site.Domain interface allowing change and expiry

* Fixup seeds so they work with V2_MIGRATION_DONE=1

* Update Sites.Cache so it's capable of multi-keyed lookups

* Implement worker handling domain change expiration

* Implement domain change UI

* Implement transition period for public APIs

* Exclude v2 tests in primary test run

* Update lib/plausible_web/controllers/site_controller.ex

Co-authored-by: Vini Brasil <vini@hey.com>

* Update lib/plausible_web/controllers/site_controller.ex

Co-authored-by: Vini Brasil <vini@hey.com>

* Update moduledoc

* Update changelog

* Remove remnant from previous implementation attempt

* !fixup

* !fixup

* Implement domain change via Sites API

cc @ukutaht

* Update CHANGELOG

* Credo

* !fixup commit missing tests

* Allow continuous domain change within the same site

---------

Co-authored-by: Vini Brasil <vini@hey.com>
4 years ago
hq1 5ca53a70be
Start :ssl for migrations (#2818)
* Start :ssl for migrations

* Start :ch in migrate.sh
4 years ago
hq1 70a85c3f4c
Revert "Temporarily disable site creation (#2813)" (#2817)
This reverts commit b626329ec5.
4 years ago
Uku Taht b626329ec5
Temporarily disable site creation (#2813)
* Temporarily disable site creation

* Fix linter

* Disabled message wording
4 years ago
RobertJoonas bc835895a2
Rename dimensions script to pageview-props (#2786)
* rename dimensions script to pageview-props

* commit mix.lock change
4 years ago
Uku Taht 46048e50f7
Support multiple filters - frontend (#2773)
* Wrap Plausible.Stats.Filters with unit tests

* Parse `member` filter type

* Support for `member` filter in `aggregate_time_on_page`

* Support `not_member` filter type

* Support `matches_member` and `not_matches_member` filters

* Extract util module for React filters

* Implement Combobox from scratch with no libs

* Support multple filter clauses in combobox

* Don't use browser / os in version label

* Show highlighted option in combobox

* WIP

* Fix location filters outside filter modal

* Align open/close behaviour with react-select

* Styling updates for combobox

* Add support for wildcards in Combobox

* Implement keybindings for combobox

* Allow free choice inputs in combobox

* Rename 'Save filter' -> Apply filter

* Remove TODO comment

* Clean up some rebase mistakes

* Rename `allowWildcard` -> `freeChoice`

* Dark mode fixes

* Remove hint from filter modal

* Escape pipe character in filter modal

* Do not allow selecting duplicate options in combobox

* Escape brackets in `page_regex/1`

* Fix disabled style in dark mode

* Add regex fallback for safari

* Show no matches found when visibleOptions is empty

* Disable enter key when no visible options

* Do not submit empty form fields

* Remove unnecessary setOpen(true)
4 years ago
hq1 d2f2c69387
Conditionally support switching between v1 and v2 clickhouse schemas (#2780)
* Remove ClickhouseSetup module

This has been an implicit point of contact to many
tests. From now on the goal is for each test to maintain
its own, isolated setup so that no accidental clashes
and implicit assumptions are relied upon.

* Implement v2 schema check

An environment variable V2_MIGRATION_DONE acts like
a feature flag, switching plausible from using old events/sessions
schemas to v2 schemas introduced by NumericIDs migration.

* Run both test suites sequentially

While the code for v1 and v2 schemas must be kept still,
we will from now on run tests against both code paths.
Secondary test run will set V2_MIGRATION_DONE=1 variable,
thus making all `Plausible.v2?()` checks return `true'.

* Remove unused function

This is a remnant from the short period when
we would check for existing events before allowing
creating a new site.

* Update test setups/factories with v2 migration check

* Make GateKeeper return site id along with :allow

* Make Billing module check for v2 schema

* Make ingestion aware of v2 schema

* Disable site transfers for when v2 is live

In a separate changeset we will implement simplified
site transfer for when v2 migration is complete.
The new transfer will only rename the site domain in postgres
and keep track of the original site prior to the transfer
so we keep an ingestion grace period until the customers
redeploy their scripting.

* Make Stats base queries aware of v2 schema switch

* Update breakdown with v2 conditionals

* Update pageview local start with v2 check

* Update current visitoris with v2 check

* Update stats controller with v2 checks

* Update external controller with v2 checks

* Update remaining tests with proper fixtures

* Rewrite redundant assignment

* Remove unused alias

* Mute credo, this is not the right time

* Add test_helper prompt

* Fetch priv dir so it works with a release

* Fetch distinct partitions only

* Don't limit inspect output for partitions

* Ensure SQL is printed to IO

* Remove redundant domain fixture
4 years ago
Uku Taht a75d0b35b0
Fix Timex.total_offset blowing up during clock changes (#2788)
* Fix Timex.total_offset blowing up during clock changes

* Format large numbers with underscore in tests

Co-authored-by: Adam <hq@mtod.org>

---------

Co-authored-by: Adam <hq@mtod.org>
4 years ago
Adam 6637751a5e
Implement Numeric IDs migration (#2762)
* Implement Numeric IDs migration

* Fix typo

* Mute credo for now

* Improve configurability and add stop_t

* Adjust to Ch/Chto only

* Fix opts key for dictionary password

* Add regular ecto migration with numeric ids v2 schemas (#2768)

* Add regular ecto migration

* Fix typo

* Update priv/ingest_repo/migrations/20230320094327_create_v2_schemas.exs

Co-authored-by: Vini Brasil <vini@hey.com>

* Implement v2 events/sessions schema modules (#2777)

* Implement v2 events/sessions schema modules

* Clean up session schemas

---------

Co-authored-by: Vini Brasil <vini@hey.com>

* Update moduledocs

---------

Co-authored-by: Vini Brasil <vini@hey.com>
4 years ago
Vini Brasil 87a63fe28a
Custom comparison mode (#2753)
This commit implements the custom comparison mode using a date range.
4 years ago
Marko Saric 9f71bb3921
Updating password reset copy (#2775) 4 years ago
Uku Taht a154a4214b
Support multiple filters - backend (#2749)
* Wrap Plausible.Stats.Filters with unit tests

* Parse `member` filter type

* Support escaped | in member filter

* Support for `member` filter in `aggregate_time_on_page`

* Add support for `member` filter type on goals

* Disable Credo warning

* Support `not_member` filter type

* Disable credo for `query_sessions`

* Support `matches_member` and `not_matches_member` filters

* Disable Credo for `Filters.filter_value/2`

* Support `matches_member` and `not_matches_member` for goal filter

* Support for contains_member and friends

* Updates for new chto driver

* make top_stats_test.exs:203 pass (#2779)

---------

Co-authored-by: ruslandoga <rusl@n-do.ga>
4 years ago
Adam 6d79ca5093
Switch to new clickhouse adapter (ch/chto) (#2733)
* another clickhouse adapter

* don't restore stats_removal.ex

* fix events main-graph error (#2746)

* update ch, chto

* update chto again (#2759)

* Stop treating page filter as an entry_page filter (#2752)

* remove dead code

* stop treating page filter as entry page filter in breakdown queries

* stop treating page filter as entry page filter in aggregate queries

* stop treating page filter as entry page filter in timeseries queries

* mix format

* update changelog

* break code down to smaller functions to keep credo happy

* remove unused functions

* make CSV export return only conversions with goal filter (#2760)

* make CSV export return only conversions with goal filter

* update changelog

* update elixir version in mix.exs (#2742)

* revert admin.ex changes (#2776)

---------

Co-authored-by: ruslandoga <67764432+ruslandoga@users.noreply.github.com>
Co-authored-by: ruslandoga <rusl@n-do.ga>
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
4 years ago
Adam 736e6e385c
Capture no_cache sentry messages together (#2772) 4 years ago
RobertJoonas 4175a7abae
make CSV export return only conversions with goal filter (#2760)
* make CSV export return only conversions with goal filter

* update changelog
4 years ago
RobertJoonas d0895efbdd
Stop treating page filter as an entry_page filter (#2752)
* remove dead code

* stop treating page filter as entry page filter in breakdown queries

* stop treating page filter as entry page filter in aggregate queries

* stop treating page filter as entry page filter in timeseries queries

* mix format

* update changelog

* break code down to smaller functions to keep credo happy

* remove unused functions
4 years ago
Marko Saric 2204d67260
Add a note that a subscription is not transferred alongside a site (#2750)
* Add a note that a subscription is not transfered alongside a site

* Update transfer_ownership_form.html.eex

* Format membership/transfer_ownership_form.html.eex

---------

Co-authored-by: Vinicius Brasil <vini@hey.com>
4 years ago
Marko Saric b17710a706
Removing the notes about the 48 hour deletion delay (#2735)
* Update site.ex

* Update settings_danger_zone.html.eex
4 years ago
Uku Taht 8c8cf23bf0
Embedded dashboard fixes (#2744)
* Add padding to embedded dashboards

* Changelog entry

* Do not check origin in iframe-resizer
4 years ago
Adam 3417b2dd0e
Disallow site creation with insufficient input (#2741) 4 years ago
RobertJoonas ad3edbfb9a
Bug fix - show `views_per_visit` when stats include imported data (#2738)
* show views_per_visit with imported data

* use optional chaining instead
4 years ago
RobertJoonas 4eca1aef70
fix a bug in percent change calculation (#2731) 4 years ago
Adam 4b21b4e6d0
Remove Firewall plug; redundant at infra level (#2730)
* Remove Firewall plug; redundant at infra level

* Update changelog
4 years ago
Vini Brasil 8b0f0cabc2
Implement "Year over Year" comparison mode (#2704)
This pull request adds support for multiple comparison modes, changes the comparison checkbox to a combobox, and implements the year over year comparison mode. The feature is still behind a feature flag.

Co-authored-by: Uku Taht <uku.taht@gmail.com>
4 years ago
RobertJoonas 874d664521
Add the new `Views per Visit` metric to dashboard Top Stats and CSV export (#2728)
* globally rename 'pages_per_visit' to 'views_per_visit'

* change the order of top stats

* rename 'Visits' to 'Total visits' in the UI

* add views_per_visit to UI

* put the new metric under a feature flag

* add new metric to CSV export under feature flag

* mix format

* use only one feature flag
4 years ago
RobertJoonas 0d6c72d50f
Add new `Pages / Visit` metric to Stats API Aggregate and Timeseries (#2714)
* refactor metric validation

* link to the correct docs section

* add the new metric to aggregate API

* explicitly remove __internal_visits

The overall metric selection is well defined by
`Plausible.Stats.Timeseries.empty_row/2`. The only metric that needs
to be removed from the timeseries response is __internal_visits.

This commit also moves the `remove_internal_visits_metric` function to a
new Util module to be used by both breakdown and timeseries.

* add the new metric to timeseries API

* mix format

* add moduledoc to keep credo happy

* convert pages_per_visit to string straight away

* do rounding in db query

* query # of visits with sum(sign) instead

* stop converting pages_per_visit to string
4 years ago
Uku Taht 43bf7dd09f
Use user-agent instead of screen_width to get device type (#2711)
* Use user-agent instead of screen_width to get device type

Co-authored-by: eriknakata <erik.nakata5@gmail.com>

* Fix credo

* Log on unhandled UAInspector device type

* Make 'browser' the default tab in devices report

* Remove device tooltip

* Remove screen_width from ingestion completely

* Remove browserstack harness, run playwright directly

* Select meta key based on OS platform

* Run CI tests in parallel

* Improve device match readability

* Add changelog

---------

Co-authored-by: eriknakata <erik.nakata5@gmail.com>
4 years ago
Uku Taht f26ca7da9f
Update native stats start date when transferring (#2719) 4 years ago
Adam 8f86036e57
Keep track of native stats start timestamp when retrieving data (#2715)
* Stats boundary/PoC?

* Delete stats removal

* Drop events check on site creation

* Update seeds script

* Use native_stats_start_at

* Don't rely on native stats pointer in imported stats queries

* Reset site

* Export reset/1

* Remove unnecessary inserted_at settings

* Update seeds

* Remove unnecessary inserted_at setting
4 years ago
Adam Rutkowski 2d8775e5ef
Add extra error info + telemetry for AsyncInsertRepo (#2708)
* Add extra error info + telemetry for AsyncInsertRepo

* format
4 years ago
Adam Rutkowski 867dad6da7
Implement ingest counters (#2693)
* Clickhouse migration: add ingest_counters table

* Configure ingest counters per MIX_ENV

* Emit telemetry for ingest events with rich metadata

* Allow building Request.t() with fake now() - for testing purposes

* Use clickhousex branch where session_id is assigned to each connection

* Add helper function for getting site id via cache

* Add Ecto schema for `ingest_counters` table

* Implement metrics buffer

* Implement buffering handler for `Plausible.Ingestion.Event` telemetry

* Implement periodic metrics aggregation

* Update counters docs

* Add toStartOfMinute() to ordering key

* Reset the sync connection state in `after` clause

* Flush counters on app termination

* Use separate Repo with async settings enabled at config level

* Switch to clickhouse_settings repo root config key

* Add AsyncInsertRepo module
4 years ago
Adam Rutkowski 7dec454a23
Don't rely on Mix.env check in runtime (#2703) 4 years ago
RobertJoonas bc557b78fd
Add visits metric under a feature flag (#2687)
* Add visits metric and make it graphable

* include visits metric in csv export (visitors.csv)

* put visits under a feature flag (CSV export)

* feature flag for displaying visits on the dashboard

* fix formatting

* add visits metric to top stats (fix)

* fix imported_test to expect visits metric included

* fix formatting
4 years ago
Adam Rutkowski ecb552479f
Upgrade erlang/elixir/alpine stack (#2695)
* Dockerfile: pin elixir-1.14.3, erlang-25.2.3, alpine-3.17.0

* asdf: pin erlang 25.2.3, elixir 1.14.3-otp-25

* Remove nonsense

* Remove erlang dupe

* Remove erlang, it's in the base image already

* Remove empty file

* Truncate seconds in NaiveDateTime instances

* Bump cache key

* Mitigate Error:error:0308010C:digital envelope routines::unsupported

See: https://www.newline.co/@kchan/how-to-fix-the-error-errorerror0308010cdigital-envelope-routinesunsupported--0f8d3f17

* Upgrade to latest alpine image and pin to an exact digest

* Revert "Upgrade to latest alpine image and pin to an exact digest"

This reverts commit cdf6624efa.

* Pin alpine image to an exact digest

---------

Co-authored-by: Cenk Kücük <c@cenk.me>
4 years ago
Uku Taht 3ed5e9d27d
Refactor visits metric internals (#2691)
* Stop relying on implicitly selected `visits` metric

* Make the `visits` metric more explicit in imported data queries

* Rename internally used `visits` to `__internal_visits`
4 years ago
Uku Taht 531dfb114b
Refactor: Use HeadlessUI for search-select component (#2676)
* Use HeadlessUI for search select box

* Remove downshift from package.json

* More consistent API for Combobox component

* Combine toFilterType and valueWithoutPrefix into a single function

* Rename MyCombobox -> PlausibleCombobox

* Update webpack-cli

* Disable cache for build

* Revert "Disable cache for build"

This reverts commit aa130541f8.

* Disable cache for build

* Update webpack dependencies

* Remove glob from webpack config

* Webpack is required by package.json

* Require autoprefixer in postcss config

* Revert build changes

* Fix styling for dark mode
4 years ago
Uku Taht 412e8df41b
Validate property key in breakdown API endpoint (#2686)
* Validate property key in breakdown API endpoint

* Add missing properties

* Do not allow empty custom prop
4 years ago
Vini Brasil 535874be6f
Reject unknown imported cities from queries (#2675)
* Reject unknown imported cities from queries

This commit fixes a bug where the city report returned `N/A` entries.
The functions that build imported data queries were using SQL
`COALESCE`, assuming city data is `NULL` when unknown, when actually its
unknown value is `0`.

This commit addresses the problem using SQL `NULLIF` combined with the
previous `COALESCE` call. With this change both `0` and `NULL` are
treated as unknown.

Since 1cb07efe6d cities can be `NULL`, but
previously we saved `0` as unknown.

Closes #1960

* Add entry to CHANGELOG

* Ignore cyclomatic complexity Credo check
4 years ago
Vini Brasil 1cb07efe6d
Save city name when importing from GA (#2608)
This commit adds city data to imported records from Google Analytics. The
current implementation sets city to 0 because GA does not use the GeoNames
database.

Google Analytics Reporting API uses [Geographical IDs](https://developers.google.com/analytics/devguides/collection/protocol/v1/geoid)
to identify cities and countries. Plausible uses
[GeoNames](https://geonames.org/) and I couldn't find databases corelating the
two.

Fortunately, GA also returns the city name and this commit uses the city name
and the country ISO code to find the Geoname ID. To avoid making expensive ETS
searches, I created another ETS table in the Location library that uses
{country, city} as a key.

Related PR: https://github.com/plausible/location/pull/3
4 years ago
Uku Taht 8fcf4d3304
Fix 'field key does not exist' error (#2674) 4 years ago
ruslandoga e9ba60c8b4
improve transfer ownership error message (#2651)
* improve transfer ownership error message

* add changelog

* simplify

* revert changeset invitation error message

* more descriptive error message

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Adam Rutkowski 8f85b110aa
Split Clickhouse pools into Read-Only and Read/Write (dedicated to writes) (#2661)
* Configure ingest repo access/pool size

If I'm not mistaken 3 is a sane default, the only
inserts we're doing are:

  - session buffer dump
  - events buffer dump
  - GA import dump

And all are serializable within their scopes?

* Add IngestRepo

* Start IngestRepo

* Use IngestRepo for inserts

* Annotate ClickhouseRepo as read_only

So no insert* functions are expanded

* Update moduledoc

* rename alias

* Fix default env var value so it can be casted

* Use IngestRepo for migrations

* Set default ingest pool size from 3 to 5

in case conns are restarting or else...

* Ensure all Repo prometheus metrics are collected
4 years ago
Uku Taht 4494252cce
Add data-domain to tracking script (#2663) 4 years ago
Uku Taht 3b73836136
Fix unknown bind meta (#2654)
* Fix 'unknown bind meta' error

* Changelog

* Use one case instead of nested ifs

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Uku Taht 1cc5ac0b11
Fix bug with multiple ARRAY JOIN query (#2653)
* Fix bug with multiple ARRAY JOIN query

* Add changelog entry

* Remove 'bug' label from test description

* Simplify pattern match

Co-authored-by: Adam Rutkowski <hq@mtod.org>

* Rename has_join_with_table -> joins_table?

Co-authored-by: Adam Rutkowski <hq@mtod.org>

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
ruslandoga c3e736aae6
add flash ttl (#2652) 4 years ago
Vini Brasil 5159e683cc
First pass at Comparisons (#2650)
This commit adds support for comparing the actual showed period on the main graph with the previous one. This is a first pass and it's hidden under a feature flag because it's not feature complete yet as we want to support other comparison modes.
4 years ago
Adam Rutkowski 8a2df294f5
Replace empty OS/screen size/Browser with `(not set)` in UI/API/exports (#2646)
* Make Device section components aware of (not set)

So that no extra sub-filters are possible when the unset
top item is selected.

* Support '(not set)' in breakdown/filters

* Update expectations for export tests

* Add extra tests for returning/filtering by '(not set)'

* Add changelog entry

* Remove ListReport conditional render

* Prevent redundant sub-filters

* Fix filter text rendering

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
4 years ago
Marko Saric 4ea519fd11
Some changes to the "site taken" copy (#2647)
* Some changes to the "site taken" copy

* Update site.ex

* Fix tests

---------

Co-authored-by: Vinicius Brasil <vini@hey.com>
4 years ago
Adam Rutkowski 482249ffba
Better error message on site creation error (#2644)
* Ensure clear error message on site creation error

* Update colons in API error messages
4 years ago
Adam Rutkowski f678cfab25
Prevent error reports from being sent with short feedback (#2635) 4 years ago
Adam Rutkowski 8f9f032968
Delay stats deletions (#2632)
* Implement Site removal transaction

* Implement Stats removal Oban worker

* Configure site removal queue

* Call Site.Removal.run() instead of Purge.delete_site!

* Test site/stats removal

* Remove FIXME - filed a ticket

* Over-communicate lenghty deletion process to the users
4 years ago
Marko Saric 66c65e0071
Changes to the site settings danger zone (#2629)
* Changes to the site settings danger zone

The idea was to:

1. Show a transfer site ownership message to owners of the site only. This will make ownership transfer feature more discoverable. Will also help people to transfer ownership of a site rather than deleting the whole thing when that makes more sense
2. Add a note that site deletion is a delayed operation
3. Make some other tiny copy changes

What did I break? :)

* Fix markup so the long text wraps sooner

* Fix indent

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Adam Rutkowski ad12e1ef31
Show user feedback form on server errors (#2617)
* Move Endpoint errors setup to common config

* Implement naive Sentry link resolver

* Implement error report e-mail

* Delete static sentry script

* Implement user feedback form on server errors

* Re-arrange pipe

* Use Sentry.Config.dsn() where applicable

* Fix typo

* Use Map.replace/3
4 years ago
Marko Saric c1d174d0fa
Some changes to the "locked site" copy (#2615)
Some changes to be more consistent with the emails we send. Also "valid" subscription rather than "active" subscription fits better for the different cases where this screen is shown
4 years ago
RobertJoonas bd0de97521
Fix returning more pageviews with a visit property filter (#2612)
* fix subquery for sessions in base_event_query/2

As the 'sessions' table is using the CollapsingMergeTree engine, we have
to select session_id's distinctively. Otherwise we will get multiple rows
(with sign -1 and 1) as long as the background merge hasn't happened.

* update changelog

* use GROUP BY instead of SELECT DISTINCT

* remove comma
4 years ago
ruslandoga e3f095ade7
fix class name typo and include it in safelist (#2610) 4 years ago
Adam Rutkowski 1a4b65c36c
Communicate site creation edge cases to the users (#2605)
* Fingerprint DBConnection.ConnectionError in Sentry

* Check events before creating a site

* Enable sites limit screen

* Remove debugging remnant

* Fix buggy assertions

This wasn't doing what expected:

  iex(1)> Repo.exists?(Plausible.Site, domain: "foo")
  [debug] QUERY OK source="sites" db=0.6ms idle=1906.2ms
  SELECT TRUE FROM "sites" AS s0 LIMIT 1 []

* Encapsulate check to satisfy credo

* Use less technically involved error message

* Bring back e-mail to the limit error message
4 years ago
Vini Brasil 86d5098b31
Stop logging invalid_address errors from Locus (#2604)
This commit stops logging `{:invalid_address, "ip"}` errors. This
reduces noise on Sentry, that is capturing error logs. Other errors are
still logged.
4 years ago
Uku Taht 818d4404a8
Support custom events and props in mix send_pageview (#2606) 4 years ago
ruslandoga 166748dcf2
Replace Geolix with Locus (#2362)
This PR replaces geolix with locus to simplify self-hosted setup. locus can auto-update maxmind dbs which are recommended for self-hosters if they want city-level geolocation. locus is also a bit faster.

This PR also uses a test mmdb file from https://github.com/maxmind/MaxMind-DB for e2e geolocation tests without stubs.
4 years ago
Adam Rutkowski 5ba8697c31
Fingerprint DBConnection.ConnectionError in Sentry (#2597) 4 years ago
Vini Brasil cb5d8a0d25
Make warning notices responsive (#2591)
Closes #2528
4 years ago
RobertJoonas 3999f282a5
Last updated tooltip (#2576)
* extract blinkingDot function

* position pulsating-circle with tailwind instead

* remove unused function

* extract renderStatName function

* display seconds since last realtime update

Adds a 'Last updated X seconds ago' label to the Current Visitors tooltip.

* small refactor: avoid duplication of this.props and this.state

* show the 'last updated ...' tooltip in historical

* changelog update

* use className utility function
4 years ago
Adam Rutkowski 61bf624761
Sentry cleanup (#2590)
* Filter DBConnection logs for clickhouse in Sentry

Removing the never matching clause btw

* Exclude Plug.CSRFProtection.InvalidCSRFTokenError from Sentry

* Turn common Sentry captures into logged warnings

* Exclude fonts from static conf and InvalidPathError from Sentry
4 years ago
Adam Rutkowski 97a3ce72e4
Update clickhousex & fingerprint sentry errors (#2587)
* Revert "Revert "Fingerprint Clickhousex.Error exceptions in Sentry (#2578)" (#2586)"

This reverts commit a78e07236a.

* Bump clickhousex to include https://github.com/plausible/clickhousex/pull/20
4 years ago
Adam Rutkowski a78e07236a
Revert "Fingerprint Clickhousex.Error exceptions in Sentry (#2578)" (#2586)
This reverts commit 2428a2bf36.
4 years ago
Adam Rutkowski 2428a2bf36
Fingerprint Clickhousex.Error exceptions in Sentry (#2578)
* Fingerprint Clickhousex.Error exceptions in Sentry

* Update clickhousex
4 years ago
Marko Saric 934c3c323e
Some changes to the 'dashboard locked' email copy (#2584) 4 years ago
Marko Saric 5ba446e87c
Some changes to the 'over limit' email copy (#2583) 4 years ago
Adam Rutkowski 4bde884da8
Self-host Sentry user feedback script (#2579)
* Add static, minified sentry bundle

OG source: https://browser.sentry-cdn.com/5.9.1/bundle.min.js

* On 500s, load our self-hosted sentry bundle
4 years ago
RobertJoonas 13a05b4e1e
Change metric labels for realtime with goal filter (#2560)
* change metric labels for realtime with goal filter

* changelog

* fix formatting
4 years ago
Uku Taht 51fc9c0945
Refine parallel application-side joins (#2564)
* Consolidate task timeouts, incrase to 15s

* Use Task.async_stream for parallel Clickhouse queries

* Propagate Opentelemetry context to child Task process
4 years ago
Vini Brasil 4503895d0a
Fix breakdown API pagination when using event metrics (#2562)
* Fix breakdown API pagination when using event metrics

This commit fixes a bug where the subsequent breakdown API pages had
the same items as the first page. The fix sorts the underlying
ClickHouse query by timestamp, keeping the same order between requests,
as we use OFFSET/LIMIT pagination.

* Fix repeated results assertion

* Add different ORDER BY to each breakdown property
4 years ago
Adam Rutkowski 44afbaa2f9
Wrap paddle actions with DB transactions (#2558)
* Wrap paddle actions with DB transactions

* Bow to credo

* Make the paddle transactions crash when applicable

* s/changeset/details
4 years ago
Uku Taht 1785653b1e
Ignore unknown countries (#2556)
* Ignore XX and T1 countries

* Add fallback if country_code=nil

* Lookup city overrides directly in CityOverrides module

* Changelog

* Add empty moduledoc

* Remove redundant comment
4 years ago
Vini Brasil 22e2ae1844
Add OTEL tracing to breakdown functions (#2544)
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
4 years ago
Vini Brasil 10a9767912
Remove ingestion requests and Oban queries from OTEL (#2543) 4 years ago
Vini Brasil 5152e8d416
Reject events with long URIs and data URIs (#2536)
* Reject events with data URIs

* Reject events with URIs longer than 2,000 characters

* Update CHANGELOG.md
4 years ago
Vini Brasil 0e87b489f6
Return empty list when breaking down by event:page without events (#2530)
* Return empty list when breaking down by event:page without events

This commit fixes a bug with pagination where breaking down by event:page
would always return results despite pagination.

Closes #2255

* Update CHANGELOG.md
4 years ago
Uku Taht a37b3433d7
Always show direct traffic in sources reports (#2531)
* Remove show_noref behaviour

Removes query param show_noref which was used from React to control
whether to show Direct / None traffic or not. The show_noref behaviour
was untested previously.

Closes #2523

* Add changelog entry

* Fix tests

* Removed files I did not mean to check in :)
4 years ago
Vini Brasil 478e0c6990
Handle GA authentication errors (#2505) 4 years ago