Commit Graph

1245 Commits (db5488efa677563cc3fce874f72ccb4e7c939708)

Author SHA1 Message Date
hq1 db5488efa6
Put live view meta tags behind a FF (#3083) 3 years ago
hq1 0c77b01a99
Allow configurable websocket URL for live view (#3082) 3 years ago
hq1 bbedeff683
Implement Funnels view on dashboard (#3066)
* Add Funnel react component

assets/js/dashboard/stats/behaviours/funnel.js - restored from:

98a76cbd Remove console.info calls
d94db99d Convert Funnel class component into a functional one
028036ad Review comments
3067a940 Stop doing maths in react
73407cc3 Fix error handling when local storage gets corrupted
e8c6fc52 Format numbers on funnel labels
c815709f Reorganize component responsibility
7a88fe44 Outline basic error handling
94caed7c Chart styling updates
4514608a Add percentages to funnel
d622c32d Add funnel picker

Co-authored-by: Uku Taht <uku.taht@gmail.com>

* Pass funnels list to react via data-funnels

* Implement Funnels react API

lib/plausible_web/controllers/api/stats_controller.ex - restored from:

f36ad234 Adjust to Plausible.Stats interface
9b532273 Test funnel stats controller
028036ad Review comments
bea3725f Remove IO.inspect
7a88fe44 Outline basic error handling
c8ae3eaf Move Funnels to StatsController and use base query
667cf222 Put private functions at the bottom

* Tweak funnel presentation

* Handle errors at the top

* Do not register DataLabels plugin globally

or else all the existing charts are affected

* Calculate drop-off percentage evaluating funnels

* Tweak dark mode + implement nicer tooltips

* Make currently selected funnel bold in the picker

* Count user_ids not session_ids when evaluating funnels

So if a visitor goes:

1. Start session
2. Complete funnel step 1
3. Inactive for 30 minutes
4. Complete funnel step 2

We would not be able to track this funnel completion because of the session timeout.
We like to o measure this as funnel completion even though the session expired in the middle.

cc @ukutaht

* Add extra properties to the funnels API

cc @ukutaht

* Improve tooltips so that step to data is rendered

* Change tooltip number formatting

* Remove debugging remnants

* Quick & dirty mobile view

* Fix mobile view: tweak dark mode & funnel switching

* Ignore DOMException: the operation was aborted

Otherwise this sometimes flashes the space shuttle
screen when navigating quickly via a keyboard.

* Format percentages on the main chart

* Close missing tag 🙈

* Revert "Close missing tag 🙈"

This reverts commit 9c2f970e22fd7e2980503242b414f42ce8bce1d2.

* Use jsx to render funnel tooltip

To get markup validated via lsp mostly...

* Fixup: s/class/className

* Fix className interpolation

* Add a ruler to the tooltip

* Tweak funnel chart style

* Fix font distortion issue on chart/canvas labels

* s/class/className

* Put "Set up funnels" link behind a feature flag

* Refactor internal selection storage

Getting ready for live funnel evaluation

* Don't try to connect LV socket if there's no CRSF token set up

This is perfectly okay for some of the templates/layouts.

* Fix up funnel creation typespecs

Unfortunately we can't define a type with literal string keys,
hence this must suffice.

* Use uniq over count/distinct

* Revert JSX in tooltips

Ref: https://github.com/plausible/analytics/pull/3066#discussion_r1241891155

* Remove the extra query for counting all visitors

cc @ukutaht

* Add premium notice

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
3 years ago
Vini Brasil a8daac5e03
Add notice to revenue goals UI (#3069)
This pull request changes the UI for revenue goals, replaces the select box with a toggle, and adds a notice. It also polishes up the goal creation UI in general to have the same font sizes and weights compared to the rest of the page.
3 years ago
Vini Brasil b97290b5cf
Add revenue metrics to top stats (#3059)
This commit adds revenue data to top stats. Average and total are displayed when filtering by a revenue goal (or many if they have the same currency set).
3 years ago
hq1 d8543c81cc
Funnel site settings (#3039)
* Update formatter config

* Install LiveView JS integration & hooks

* Temporarily update endpoint/session config

* Optionally allow preloading funnels for goals

* Site controller

* Implement funnel settings

lib/plausible_web/live/funnel_settings/combo_box.ex - restored from:

054de6e2 Fix the tab/blur bug again
20da4c89 Rename InputPicker to ComboBox

lib/plausible_web/live/funnel_settings/form.ex - restored from:

9bedda3b Remove potential FIXME
20da4c89 Rename InputPicker to ComboBox
028036ad Review comments
aea4ebc4 Access Funnel min/max steps via the __using__/1 macro
0dde27fd Remove inspect call
eed588a7 Start testing the funnel editor
0e95228b Extract funnel settings test module
7b16ace5 Leverage aplinejs to deal with the tyranny
8dc6a3e7 wip
cf228630 wip
30a43fd1 wip
89f10ecb wip
950a18d9 Dirty funnel save
298a6a53 wip
7690d50f wip
639c6238 fixup
aa59adeb wip
ff75c00b wip

lib/plausible_web/live/funnel_settings/list.ex - restored from:

4eae122c Fix data-confirm attr interpolation
51f0397d Implement deleting funnels
1f6fe25d Add number of steps to funnels list
298a6a53 wip
ff75c00b wip

test/plausible_web/live/funnel_settings/funnel_settings/combo_box_test.exs - restored from:

20da4c89 Rename InputPicker to ComboBox

test/plausible_web/live/funnel_settings/funnel_settings_test.exs - restored from:

34822ff4 Bootstrap InputPicker tests

lib/plausible_web/live/funnel_settings.ex - restored from:

028036ad Review comments
acd9c4f2 Prepare ephemeral funnel definitions so that users can test funnels
51f0397d Implement deleting funnels
0e95228b Extract funnel settings test module
8dc6a3e7 wip
89f10ecb wip
950a18d9 Dirty funnel save
298a6a53 wip
aa59adeb wip
ff75c00b wip

test/plausible_web/controllers/error_report_controller_test.exs - restored from:

34822ff4 Bootstrap InputPicker tests

test/support/html.ex - restored from:

0a53979d Improve InputPicker tests - include AlpineJS assertions
34822ff4 Bootstrap InputPicker tests

lib/plausible_web/views/layout_view.ex - restored from:

b490403b !ifxup

lib/plausible_web/templates/site/settings_funnels.html.eex - restored from:

51f0397d Implement deleting funnels
ea1315f3 Test funnels list in settings
7b16ace5 Leverage aplinejs to deal with the tyranny
ff75c00b wip
4da25c35 Fixup

lib/plausible_web/templates/layout/app.html.eex - restored from:

ff75c00b wip

* Add funnel settings route

* Warn about funnels deletion when deleting goals

lib/plausible_web/templates/site/settings_goals.html.eex - restored from:

fdd9bcd0 Fixup
f1e6364d Merge remote-tracking branch 'origin/master' into funnels-rebase
9d0b7c6d Fix markup error
4a4ddbdc Optionally preload funnels for goals and stub funnel-goal deletion
ebdc4333 Extend the prompt in case of funnel-goal deletion
639c6238 fixup
aa59adeb wip

* Split new JS LiveView additions

* Put funnels behind a feature flag

* Integrate dashboard feature toggle

* Update signing salt for live view

* Update moduledocs

* Update live reloader config

* Use Phoenix.HTML.Safe for goal names

* Workaround to get flashes working in embedded liveview

* Keep feature toggles idempotent, rename property to setting

We'll still retain the ability to flip bools on a lower level.

* Update moduledocs

* Make live flash disappear after 5s

* Tailwind: purge .heex files too

* Update docs link

* Add live components to tailwind purge config

* Update another flaky test

Ref f0bdf872
cc @vinibrsl

* Fix combobox input length w/ WebKit

* Intoduce generic notice component

* Revert "Fix combobox input length w/ WebKit"

This reverts commit 3c653a6d85d5000167631e10ef45a93c13b41ed1.

* Fix combobox input length on webkit

* Make whole combobox item clickable, not only text

* Fix glitch moving Save button on activation

* Tweak dark mode

* Show funnel form without waiting for funnel name input

* Tweak dark mode

* Include static Phoenix components in tailwind purge

* Tune funnels form into a liveview of its own

This is so that ComboBoxes can publish their selections
and unavailable choices can be propagated to other siblings.

* Push less data over websocket

* Undo Lsp/formatter race condition

* Fixup typespecs

* Bust CI cache
3 years ago
Vini Brasil 81444f077d
Remove revenue metrics from pageview goal query (#3040)
This commit fixes a bug where performing a breakdown by goal with
revenue metrics would return data for both event goals and pageview
goals. However, revenue data is only applicable for event goals as they
hold currency data.

To fix the bug, this commit removes revenue metrics from the pageview
breakdown query.
3 years ago
Vini Brasil 5012e0c0ec
Set with_imported=false in realtime mode (#3053)
This commit fixes a bug where timeseries queries - such as the main
graph - would fail with imported data in realtime mode. Realtime mode
`date` field is not actually a date, but minutes from now. This would
cause the imported tables join to fail with this error:

```
(Ch.Error Code: 53. DB::Exception: Can't infer common type for joined
columns: date: Int64 at left, s1.date: Date at right. There is no
supertype for types Int64, Date because some of them are
Date/Date32/DateTime/DateTime64 and some of them are not.
(TYPE_MISMATCH)
```

This commit removes imported data from realtime queries, as it doesn't
make sense to include it. Imported data does not have time precision,
and would only appear in the first day the data was imported anyways.
3 years ago
Vini Brasil 8834486a19
Pass cached site struct down the ingestion pipeline (#3027)
* Pass cached site struct down the ingestion pipeline

Revenue goals need the cached site struct during ingestion to get the
goals name and currency. This cache lookup is not necessary as
`GateKeeper.check/1`, which is called first in the ingestion pipeline,
could already return the site struct from the cache.

This commit changes `GateKeeper.check/1` to return the site struct
instead of the site ID. Moreover, this commit changes the ingestion
pipeline to avoid calling the sites cache twice.

Related: https://github.com/plausible/analytics/pull/2957#discussion_r1203921549

* Remove revenue_goals unnecessary fallback

* Change duplicate child_id in cache test

* Remove revenue goal condition from cache query

* Remove Plausible.DataCase.reload/1
3 years ago
Uku Taht 98055d2f1c
Fix queries with wildcard goals (#3015)
* Fix queries with wildcard goals

* Changelog
3 years ago
Vini Brasil 8f6224b5de
Revenue tracking: goal conversions and tracker script (#3019)
* Add revenue average and total to Goal Conversions

* Add revenue option to tracker script

* Simplify revenue tracker script
3 years ago
ruslandoga fd15853965
disable registration by default in self-hosted setups (#3014)
* disable registration by default in self-hosted setups

* add changelog entry

* add error flash
3 years ago
Vini Brasil 2c4cea6dce
Add revenue metrics in StatsController instead of Stats.Breakdown (#3028)
This commit makes total_revenue and average_revenue first-class metrics
instead of adding them magically in `Plausible.Stats.Breakdown`. To
keep the same behavior, revenue metrics are added in StatsController.

Related: https://github.com/plausible/analytics/pull/2957#pullrequestreview-1475017620
3 years ago
hq1 a356009af8
Implement backend core for funnels (#3023)
* Implement Funnel schema

* Implement Funnels context module

lib/plausible/funnels.ex - restored from:

dca93621 Mark Vini's suggestion applied
7d1eec75 Update lib/plausible/funnels.ex
028036ad Review comments
b694ccd5 Express the guard nicely
acd9c4f2 Prepare ephemeral funnel definitions so that users can test funnels
aea4ebc4 Access Funnel min/max steps via the __using__/1 macro
51f0397d Implement deleting funnels
cf228630 wip
1f6fe25d Add number of steps to funnels list
89f10ecb wip
950a18d9 Dirty funnel save
e62bfce5 Refactor funnel result evaluation
88e38a33 Update funnel tests
bf47f654 Calculate conversion rate/dropoff in the backend
9e62ffb1 Make existing tests pass
7a88fe44 Outline basic error handling
c8ae3eaf Move Funnels to StatsController and use base query
2929bfd4 Minor rework
b490403b !ifxup
ae44db77 Add Funnels.list/1 function
4f3216b3 Unique steps
6e0c3efc Ensure step ordering works
cea93ba1 It works I think
c50ac0c2 WIP
a466ec4c Dirty wip
fbd14e23 Absolute dirty wip
3ddd15ad wip
34ceb3f5 dirty wip

test/plausible_web/controllers/api/stats_controller/funnels_test.exs - restored from:

9b532273 Test funnel stats controller

Co-authored-by: Uku Taht <uku.taht@gmail.com>

* Update Goals implementation

lib/plausible/goal/schema.ex - restored from:

028036ad Review comments
acd9c4f2 Prepare ephemeral funnel definitions so that users can test funnels
4a4ddbdc Optionally preload funnels for goals and stub funnel-goal deletion
fbd14e23 Absolute dirty wip

lib/plausible/goals.ex - restored from:

aea4ebc4 Access Funnel min/max steps via the __using__/1 macro
a418c039 Implement funnel reduction logic on goal deletion
4a4ddbdc Optionally preload funnels for goals and stub funnel-goal deletion
ebdc4333 Extend the prompt in case of funnel-goal deletion
950a18d9 Dirty funnel save
f06c3fcf wip

test/plausible/goals_test.exs - restored from:

a418c039 Implement funnel reduction logic on goal deletion
4a4ddbdc Optionally preload funnels for goals and stub funnel-goal deletion
dee5bc9e Fixup tests

* Update seeds

* Split funnel schema modules

* Add moduledoc to Funnel schema

* Update Funnel.Step moduledoc

* Add typespecs to Plausible.Funnels

* Implement Phoenix.HTML.Safe for Funnel schema

* Fixup a test after ordering goals by desc:id

test/plausible_web/controllers/api/stats_controller/conversions_test.exs - restored from:

26a493cb Fixup a test after ordering goals by desc:id

* Annotate Goal schema with type t()

* Reword schema module docs

* Fix up typespecs

* Stop tagging Funnels.get/2 result

* Nuke Goals.by_id!/2

* Test Funnels module

test/plausible/funnels_test.exs - restored from:

b7a6d3ad ya, rly
028036ad Review comments
acd9c4f2 Prepare ephemeral funnel definitions so that users can test funnels
51f0397d Implement deleting funnels
1f6fe25d Add number of steps to funnels list
dee5bc9e Fixup tests
950a18d9 Dirty funnel save
88e38a33 Update funnel tests
bf47f654 Calculate conversion rate/dropoff in the backend
9e62ffb1 Make existing tests pass
2929bfd4 Minor rework
b490403b !ifxup
ae44db77 Add Funnels.list/1 function
4f3216b3 Unique steps
6e0c3efc Ensure step ordering works
cea93ba1 It works I think
c50ac0c2 WIP
a466ec4c Dirty wip
34ceb3f5 dirty wip

Co-authored-by: Uku Taht <uku.taht@gmail.com>

* Apply changes related to 242081e5

* Evaluate funnels through `Stats.funnel/3` entry point

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
3 years ago
hq1 c7b8a8ef27
Bugfix for #3005 (#3025)
* Revert "Revert "UI groundwork: Conversions to Behaviors (#3005)" (#3024)"

This reverts commit dc0853bac7.

* Move ref back to LazyLoader container
3 years ago
hq1 dc0853bac7
Revert "UI groundwork: Conversions to Behaviors (#3005)" (#3024)
This reverts commit ba19f9530e.
3 years ago
RobertJoonas ba19f9530e
UI groundwork: Conversions to Behaviors (#3005)
* copy relevant files from b2ace16540

* make it work and set site.funnels to empty list

* make Behaviours a functional component

* add UI for a setup hint and always display conversions by default

* cherry-pick migration commit

* update site schema with new fields

* backend: implement disable-feature action

* switch between tabs in the behaviors section

Introduces template components to build props and funnels on. Both
only show a setup notice atm, and both are behind feature flags.

* extend API for disabling props and funnels

* render feature setup note directly from the Behaviours component

* fix UI behavior when features are hidden

* update setup notices

* add conversions feature switch to Site Settings > Goals

* mix format

* remove IO.inspect

* change setup notice - use buttons + popup confirmation

* optimize for light mode

* restrict access to setup notices

* some styling improvements

* allow super-admins to enable/disable features

* only show conversions (last 30min) in realtime mode

* use shorter display names for tabs

* optimize for mobile screens

* note about sending custom events

* changelog update + fix CI

* change HTTP verb for the disable-feature action

* change UI label for show/hide goals
3 years ago
Vini Brasil e4d4f7d954
Revenue tracking: Ingestion and breakdown queries (#2957)
* Add revenue fields to ClickHouse events

This commit adds 4 fields to the ClickHouse events_v2 table:

* `revenue_source_amount` and `revenue_source_currency` store revenue in
  the original currency sent during ingestion

* `revenue_reporting_amount` and `revenue_reporting_currency` store
  revenue in a common currency to perform calculations, and this
  currency is defined by the user when setting up the goal

The type of amount fields is `Nullable(Decimal64(3))`. That covers all
fiat currencies and allows us to store huge amounts. Even though
ClickHouse does not suggest using `Nullable`, this is a good use case,
because otherwise additional work would have to be done to
differentiate missing values from real zeroes.

I ran a benchmark with the data pattern we expect in production, where
we have more missing values than real decimals. I created 100 million
records where 90% of decimals are missing. The difference between the
tables in storage is just 0.4Mb.

* Add revenue parameter to Events API

This commit adds support for sending revenue data in ingestion using the
`revenue` parameter - aliased to `$`.

* Add revenue parameter to mix send_pageview

* Add average and total revenue to breakdown queries
3 years ago
ruslandoga b0c0214b6b fix twitter link 3 years ago
Vini Brasil c559370201
Trim prop list to 30 items during ingestion (#3009)
This commit adds an upper bound limit to custom props. It fails silently
and trims the list instead of dropping the event.
3 years ago
ruslandoga 131c99c69e
remove custom_dimension_filter feature and views_per_visit_metric flags (#2996)
* remove custom_dimension_filter feature flag

* add changelog entry

* fix shouldLinkToFilterModal removal

* remove views_per_visit_metric flag
3 years ago
Uku Taht 995556e86f
Enable sampling in the Stats API (#2985) 4 years ago
hq1 49ca6b859a
Prepare base for the funnel PRs merge (#2993)
* Update dependencies as per the funnels branch

Co-authored-by: Uku Taht <uku.taht@gmail.com>

* Mute sigil_E deprecation errors

* Use Phoenix.Component in PlausbileWeb

* Add tooltip to react/no-unknown-property whitelist

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
Co-authored-by: Vinicius Brasil <vini@hey.com>
4 years ago
ruslandoga 54f5718ac8
remove fetch.min.js shim from waiting_first_pageview page (#2999) 4 years ago
Uku Taht fc918081b5
Fix event name validation (#2975)
* Fix event name validation

* Add @moduledoc

* Move tests to Ingestion.RequestTest

* Remove comment
4 years ago
ruslandoga 25f95ae84e
add landing page (#2989)
* add landing page

* cont.

* add redirect to /sites test

* add changelog entry

* adapt for mobile
4 years ago
RobertJoonas 6c907cbf42
Fix favicon request URLS (#2976)
Also use the favicon.ex proxy in referrer drilldown
4 years ago
ruslandoga 40e95ffd3d
add LOG_FAILED_LOGIN_ATTEMPTS (#2936)
* add failed login logs

* put failed login attempt logs behind a config option

* add changelog entry

* add config test

* add auth_controller tests

* move tests to separate non-async test module

---------

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
4 years ago
ruslandoga 447ed12155
use ecto_ch (#2890)
* use ecto_ch

* use Ch type in BoolUInt8

* update ecto_ch and ch to latest versions
4 years ago
RobertJoonas dafb60164a
Allow setting sample_threshold in dashboard API queries (#2958)
* refactor Query.from/2

* allow specifying sample_threshold number in query
4 years ago
Uku Taht 6be67bc6c2
Carry _sample_factor from right side to left in join (#2962) 4 years ago
Uku Taht 1a94456586
Wrap user deletion in DB transaction (#2964) 4 years ago
Vini Brasil 10d9e3b083
Revenue tracking: Add currency field to goal creation (#2948)
* Add revenue goal option to goal creation

This commit adds a currency field to the goals form. Goals that have a
currency set are now revenue goals, and are cached with sites to later
be used during ingestion.

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

* Enable feature flag in tests

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
4 years ago
Uku Taht ca3e2e5121
Add unique constraint to api keys (#2931)
* Add unique constraint to api keys

* Fix test

* Remove `user_id` from api key index
4 years ago
Marko Saric 3482337348
link to the WP plugin and the integration guides (#2927) 4 years ago
Marko Saric 1723d3ca68
Keep the in-app footer consistent with the main site (#2944) 4 years ago
Uku Taht 2259651b21
Ignore missing subscription on update (#2930) 4 years ago
Harry Vangberg 34a6b984c7
Escape domain when creating URL for favicon (#2942)
* Escape domain when constructing favicon URL

A domain may include a slash, and in that case the domain must be
escaped, before it is used as an attribute for the image tag.

* match with 'conn.request_path' instead + test

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
4 years ago
RobertJoonas fa54efbc6d
Support `with_imported=true` in Stats API aggregate (#2919)
* default to source_query.include_imported before true

This fixes a bug where a Stats API aggregate query was trying
to query current period *without imported data* but previous
period *with imported data*.

* remove __internal_visits at a better time

As soon as we have made the db query, we don't need it anymore

* disallow events metric with imported data

* update changelog

* keep default in a better place

* remove unused option

* update test description

Co-authored-by: Vini Brasil <vini@hey.com>

---------

Co-authored-by: Vini Brasil <vini@hey.com>
4 years ago
Vini Brasil f29d07a49e
Add tracking back to upgrade-success page (#2938) 4 years ago
Vini Brasil 4ca2e26b9c
Limit ingestion payload size (#2904)
This pull request adds constraints on user input values during ingestion. The purpose of this PR is to filter out unrealistic or invalid inputs. More comments about each limit inline.
4 years ago
RobertJoonas 00d05b2edd
Fix a bug with query bindings (#2932) 4 years ago
Marko Saric 42f88be12a
Tweaks to the email copy (#2933)
* Removing the spike line

* Update dashboard_locked.html.eex
4 years ago
Uku Taht e954f946e1
Adds `nofollow` to X-Robots-Tag header in stats page (#2929) 4 years ago
RobertJoonas db359c2a6f
Filter out garbage props by `site.allowed_event_props` (#2907)
* add migration

* add not null constraint

* remove not null constraint and default value

* add site schema change

* filter garbage props for prop_key suggestions

* filter garbage props returned by StatsController.conversions/2

* add crm action to set prop allowlist

* use nullable allowed_event_props field instead

* filter garbage props in db query instead

* implement suggestion dealing with string input

* refactor the allowlist condition in db query

* improve test
4 years ago
Uku Taht eec842c46b
Remove `visits_metric` feature flag (#2898)
* Remove `visits_metric` feature flag

* Fix credo warning
4 years ago
Vini Brasil 257fa16cdc
Simplify Phoenix error template (#2913)
* Simplify Phoenix error template

* Test tracking script is not included in error pages

* Test tracking script is not rendered in error templates

* Rename error layout and remove unnecessary HTML boilerplate

* Add layout setting to errors rendered without exceptions

* Add skip_plausible_tracking option to more pages
4 years ago
RobertJoonas d161c1be0b
Upgrade phoenix (#2902)
* upgrade phoenix

Co-authored-by: Vini Brasil <vini@hey.com>

* fix a test (flash message)

The flash message in focus.html.eex was not covered by any test. This
commit fixes also fixes that.

* change function name

* remove unnecessary formatter and format

* update CI cache

* fix dialyzer error

---------

Co-authored-by: Vini Brasil <vini@hey.com>
4 years ago
hq1 6a831d9898
Harden API input validation (#2869)
* Reject invalid country codes early

* Validate all date parameters early

* Ensure request JSON parses to a map

* Re-arrange function calls

* Extend malicious filter validation
4 years ago
Vini Brasil fecbc8b455
Fix conversions top stats bug when comparing (#2893)
This commit fixes a bug where unique conversions and the unique visitors
of the previous period in comparison view were always the same.

First reported at: https://github.com/plausible/analytics/discussions/532#discussioncomment-5760187
4 years ago